2026-02-13T02:13:46.621 INFO:root:teuthology version: 1.2.4.dev9+ga2c505dae 2026-02-13T02:13:46.640 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-13T02:13:46.721 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47973 branch: wip-pdonnell-testing-20260210.171210 description: fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/suites/dbench}} email: pdonnell@redhat.com first_in_suite: false flavor: default job_id: '47973' kernel: sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: trial name: pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20260210.171210 ceph: cephfs: ec_profile: - disabled max_mds: 1 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: client: client_fscrypt_dummy_encryption: true mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - is offline because no MDS - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - BLUESTORE_SLOW_OP_ALERT - slow operation indications in BlueStore - experiencing slow operations in BlueStore - MGR_MODULE_ERROR - OSD_DOWN - osd.* is down - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - osds down - osd.* is down - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - pg .* is stuck inactive - pg .* is .*degraded - slow metadata IO - SLOW_OPS - slow request sha1: 8f8d38294e4db8750b4b1649f62ce37caa63080f subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 install: ceph: flavor: default sha1: 8f8d38294e4db8750b4b1649f62ce37caa63080f kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20260210.171210 sha1: cfcf307502511ba51c39a8b9085c2fcde505d421 owner: scheduled_pdonnell@soko04.front.sepia.ceph.com priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7166 sha1: 8f8d38294e4db8750b4b1649f62ce37caa63080f sleep_before_teardown: 0 subset: 604/1024 suite: fs suite_branch: wip-pdonnell-testing-20260210.171210 suite_path: /home/teuthworker/src/github.com_batrick_ceph_cfcf307502511ba51c39a8b9085c2fcde505d421/qa suite_relpath: qa suite_repo: https://github.com/batrick/ceph suite_sha1: cfcf307502511ba51c39a8b9085c2fcde505d421 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: trial027.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH83vhgUgJH4YFTm2SMH2SLzI0EUGuU9lcFE+A++DoWcvCUt9aqC/NJtAr4Zvf1oXa3JZTN/8xgOBYr5JJqUliA= trial028.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOJSYnchNstsWLQ2ffWdB3MzAEgCIn9PEOAoTk9sO749TvwSwX4OPI+mx3P68T2A+XT8azhDOKFDCw0Pv89X/aE= trial107.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGg9qD6zZP2W04O4LNLMDBHdl3vjw8Fk2Rr3Tz9mdCANa55HKQ0q4jmJkoyzj/3iRRqPksQ2VPniy+y7RiOwkyU= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install nvmetcli nvme-cli -y - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: distro-suffix: rockylinux-10 roleless: false skip_dashboard: true - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - ceph-fuse: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - "local kernel = py_attrgetter(yaml).get('kernel')\nif kernel ~= nil then\n local\ \ branch = py_attrgetter(kernel).get('branch')\n if branch and not kernel.branch:find\ \ \"-all$\" then\n log.debug(\"removing default kernel specification: %s\"\ , kernel)\n py_attrgetter(kernel).pop('branch', nil)\n py_attrgetter(kernel).pop('deb',\ \ nil)\n py_attrgetter(kernel).pop('flavor', nil)\n py_attrgetter(kernel).pop('kdb',\ \ nil)\n py_attrgetter(kernel).pop('koji', nil)\n py_attrgetter(kernel).pop('koji_task',\ \ nil)\n py_attrgetter(kernel).pop('rpm', nil)\n py_attrgetter(kernel).pop('sha1',\ \ nil)\n py_attrgetter(kernel).pop('tag', nil)\n end\nend\n" - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: kernel-fix-3 teuthology_sha1: a2c505dae37458c5537fa44f1320bd04eef25bb1 timestamp: 2026-02-13_01:53:26 tube: trial user: pdonnell verbose: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3932482 2026-02-13T02:13:46.721 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_batrick_ceph_cfcf307502511ba51c39a8b9085c2fcde505d421/qa; will attempt to use it 2026-02-13T02:13:46.722 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_batrick_ceph_cfcf307502511ba51c39a8b9085c2fcde505d421/qa/tasks 2026-02-13T02:13:46.722 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-13T02:13:46.723 INFO:teuthology.task.internal:Checking packages... 2026-02-13T02:13:46.723 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8f8d38294e4db8750b4b1649f62ce37caa63080f' 2026-02-13T02:13:46.723 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-13T02:13:46.723 INFO:teuthology.packaging:ref: None 2026-02-13T02:13:46.723 INFO:teuthology.packaging:tag: None 2026-02-13T02:13:46.723 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20260210.171210 2026-02-13T02:13:46.724 INFO:teuthology.packaging:sha1: 8f8d38294e4db8750b4b1649f62ce37caa63080f 2026-02-13T02:13:46.724 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20260210.171210 2026-02-13T02:13:46.840 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5241.g8f8d3829 2026-02-13T02:13:46.841 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-13T02:13:46.852 INFO:teuthology.task.internal:no buildpackages task found 2026-02-13T02:13:46.853 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-13T02:13:46.866 INFO:teuthology.task.internal:Saving configuration 2026-02-13T02:13:46.884 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-13T02:13:46.897 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-13T02:13:46.972 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial027.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47973', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-13 02:10:26.383831', 'locked_by': 'scheduled_pdonnell@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH83vhgUgJH4YFTm2SMH2SLzI0EUGuU9lcFE+A++DoWcvCUt9aqC/NJtAr4Zvf1oXa3JZTN/8xgOBYr5JJqUliA='} 2026-02-13T02:13:47.028 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial028.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47973', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-13 02:10:26.384905', 'locked_by': 'scheduled_pdonnell@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOJSYnchNstsWLQ2ffWdB3MzAEgCIn9PEOAoTk9sO749TvwSwX4OPI+mx3P68T2A+XT8azhDOKFDCw0Pv89X/aE='} 2026-02-13T02:13:47.084 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial107.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47973', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-13 02:10:26.384467', 'locked_by': 'scheduled_pdonnell@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGg9qD6zZP2W04O4LNLMDBHdl3vjw8Fk2Rr3Tz9mdCANa55HKQ0q4jmJkoyzj/3iRRqPksQ2VPniy+y7RiOwkyU='} 2026-02-13T02:13:47.084 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-13T02:13:47.096 INFO:teuthology.task.internal:roles: ubuntu@trial027.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2026-02-13T02:13:47.096 INFO:teuthology.task.internal:roles: ubuntu@trial028.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2026-02-13T02:13:47.096 INFO:teuthology.task.internal:roles: ubuntu@trial107.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2026-02-13T02:13:47.096 INFO:teuthology.run_tasks:Running task console_log... 2026-02-13T02:13:47.290 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7beef8bec860>, signals=[15]) 2026-02-13T02:13:47.290 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-13T02:13:47.301 INFO:teuthology.task.internal:Opening connections... 2026-02-13T02:13:47.301 DEBUG:teuthology.task.internal:connecting to ubuntu@trial027.front.sepia.ceph.com 2026-02-13T02:13:47.301 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T02:13:47.362 DEBUG:teuthology.task.internal:connecting to ubuntu@trial028.front.sepia.ceph.com 2026-02-13T02:13:47.363 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T02:13:47.425 DEBUG:teuthology.task.internal:connecting to ubuntu@trial107.front.sepia.ceph.com 2026-02-13T02:13:47.425 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T02:13:47.489 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-13T02:13:47.500 DEBUG:teuthology.orchestra.run.trial027:> uname -m 2026-02-13T02:13:47.516 INFO:teuthology.orchestra.run.trial027.stdout:x86_64 2026-02-13T02:13:47.516 DEBUG:teuthology.orchestra.run.trial027:> cat /etc/os-release 2026-02-13T02:13:47.571 INFO:teuthology.orchestra.run.trial027.stdout:NAME="CentOS Stream" 2026-02-13T02:13:47.571 INFO:teuthology.orchestra.run.trial027.stdout:VERSION="9" 2026-02-13T02:13:47.571 INFO:teuthology.orchestra.run.trial027.stdout:ID="centos" 2026-02-13T02:13:47.571 INFO:teuthology.orchestra.run.trial027.stdout:ID_LIKE="rhel fedora" 2026-02-13T02:13:47.571 INFO:teuthology.orchestra.run.trial027.stdout:VERSION_ID="9" 2026-02-13T02:13:47.571 INFO:teuthology.orchestra.run.trial027.stdout:PLATFORM_ID="platform:el9" 2026-02-13T02:13:47.571 INFO:teuthology.orchestra.run.trial027.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-13T02:13:47.571 INFO:teuthology.orchestra.run.trial027.stdout:ANSI_COLOR="0;31" 2026-02-13T02:13:47.571 INFO:teuthology.orchestra.run.trial027.stdout:LOGO="fedora-logo-icon" 2026-02-13T02:13:47.571 INFO:teuthology.orchestra.run.trial027.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-13T02:13:47.571 INFO:teuthology.orchestra.run.trial027.stdout:HOME_URL="https://centos.org/" 2026-02-13T02:13:47.571 INFO:teuthology.orchestra.run.trial027.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-13T02:13:47.571 INFO:teuthology.orchestra.run.trial027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-13T02:13:47.571 INFO:teuthology.orchestra.run.trial027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-13T02:13:47.572 INFO:teuthology.lock.ops:Updating trial027.front.sepia.ceph.com on lock server 2026-02-13T02:13:47.637 DEBUG:teuthology.orchestra.run.trial028:> uname -m 2026-02-13T02:13:47.652 INFO:teuthology.orchestra.run.trial028.stdout:x86_64 2026-02-13T02:13:47.652 DEBUG:teuthology.orchestra.run.trial028:> cat /etc/os-release 2026-02-13T02:13:47.707 INFO:teuthology.orchestra.run.trial028.stdout:NAME="CentOS Stream" 2026-02-13T02:13:47.708 INFO:teuthology.orchestra.run.trial028.stdout:VERSION="9" 2026-02-13T02:13:47.708 INFO:teuthology.orchestra.run.trial028.stdout:ID="centos" 2026-02-13T02:13:47.708 INFO:teuthology.orchestra.run.trial028.stdout:ID_LIKE="rhel fedora" 2026-02-13T02:13:47.708 INFO:teuthology.orchestra.run.trial028.stdout:VERSION_ID="9" 2026-02-13T02:13:47.708 INFO:teuthology.orchestra.run.trial028.stdout:PLATFORM_ID="platform:el9" 2026-02-13T02:13:47.708 INFO:teuthology.orchestra.run.trial028.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-13T02:13:47.708 INFO:teuthology.orchestra.run.trial028.stdout:ANSI_COLOR="0;31" 2026-02-13T02:13:47.708 INFO:teuthology.orchestra.run.trial028.stdout:LOGO="fedora-logo-icon" 2026-02-13T02:13:47.708 INFO:teuthology.orchestra.run.trial028.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-13T02:13:47.708 INFO:teuthology.orchestra.run.trial028.stdout:HOME_URL="https://centos.org/" 2026-02-13T02:13:47.708 INFO:teuthology.orchestra.run.trial028.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-13T02:13:47.708 INFO:teuthology.orchestra.run.trial028.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-13T02:13:47.708 INFO:teuthology.orchestra.run.trial028.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-13T02:13:47.708 INFO:teuthology.lock.ops:Updating trial028.front.sepia.ceph.com on lock server 2026-02-13T02:13:47.787 DEBUG:teuthology.orchestra.run.trial107:> uname -m 2026-02-13T02:13:47.802 INFO:teuthology.orchestra.run.trial107.stdout:x86_64 2026-02-13T02:13:47.802 DEBUG:teuthology.orchestra.run.trial107:> cat /etc/os-release 2026-02-13T02:13:47.857 INFO:teuthology.orchestra.run.trial107.stdout:NAME="CentOS Stream" 2026-02-13T02:13:47.857 INFO:teuthology.orchestra.run.trial107.stdout:VERSION="9" 2026-02-13T02:13:47.857 INFO:teuthology.orchestra.run.trial107.stdout:ID="centos" 2026-02-13T02:13:47.858 INFO:teuthology.orchestra.run.trial107.stdout:ID_LIKE="rhel fedora" 2026-02-13T02:13:47.858 INFO:teuthology.orchestra.run.trial107.stdout:VERSION_ID="9" 2026-02-13T02:13:47.858 INFO:teuthology.orchestra.run.trial107.stdout:PLATFORM_ID="platform:el9" 2026-02-13T02:13:47.858 INFO:teuthology.orchestra.run.trial107.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-13T02:13:47.858 INFO:teuthology.orchestra.run.trial107.stdout:ANSI_COLOR="0;31" 2026-02-13T02:13:47.858 INFO:teuthology.orchestra.run.trial107.stdout:LOGO="fedora-logo-icon" 2026-02-13T02:13:47.858 INFO:teuthology.orchestra.run.trial107.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-13T02:13:47.858 INFO:teuthology.orchestra.run.trial107.stdout:HOME_URL="https://centos.org/" 2026-02-13T02:13:47.858 INFO:teuthology.orchestra.run.trial107.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-13T02:13:47.858 INFO:teuthology.orchestra.run.trial107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-13T02:13:47.858 INFO:teuthology.orchestra.run.trial107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-13T02:13:47.858 INFO:teuthology.lock.ops:Updating trial107.front.sepia.ceph.com on lock server 2026-02-13T02:13:47.933 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-13T02:13:47.947 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-13T02:13:47.958 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-13T02:13:47.958 DEBUG:teuthology.orchestra.run.trial027:> test '!' -e /home/ubuntu/cephtest 2026-02-13T02:13:47.960 DEBUG:teuthology.orchestra.run.trial028:> test '!' -e /home/ubuntu/cephtest 2026-02-13T02:13:47.961 DEBUG:teuthology.orchestra.run.trial107:> test '!' -e /home/ubuntu/cephtest 2026-02-13T02:13:47.976 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-13T02:13:47.990 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-13T02:13:47.990 DEBUG:teuthology.orchestra.run.trial027:> test -z $(ls -A /var/lib/ceph) 2026-02-13T02:13:48.016 DEBUG:teuthology.orchestra.run.trial028:> test -z $(ls -A /var/lib/ceph) 2026-02-13T02:13:48.018 DEBUG:teuthology.orchestra.run.trial107:> test -z $(ls -A /var/lib/ceph) 2026-02-13T02:13:48.034 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-13T02:13:48.264 INFO:teuthology.run_tasks:Running task kernel... 2026-02-13T02:13:48.283 INFO:teuthology.task.kernel:normalize config orig: {'sha1': 'distro'} 2026-02-13T02:13:48.283 INFO:teuthology.task.kernel:config {'mon.a': {'sha1': 'distro'}, 'mon.b': {'sha1': 'distro'}, 'mon.c': {'sha1': 'distro'}}, timeout 300 2026-02-13T02:13:48.284 DEBUG:teuthology.orchestra.run.trial027:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-13T02:13:48.284 DEBUG:teuthology.orchestra.run.trial028:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-13T02:13:48.284 DEBUG:teuthology.orchestra.run.trial107:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-13T02:13:48.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T02:13:48.299 DEBUG:teuthology.orchestra.run.trial107:> uname -r 2026-02-13T02:13:48.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T02:13:48.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T02:13:48.299 DEBUG:teuthology.orchestra.run.trial028:> uname -r 2026-02-13T02:13:48.299 DEBUG:teuthology.orchestra.run.trial027:> uname -r 2026-02-13T02:13:48.354 INFO:teuthology.orchestra.run.trial107.stdout:5.14.0-645.el9.x86_64 2026-02-13T02:13:48.354 INFO:teuthology.task.kernel:Running kernel on trial107: 5.14.0-645.el9.x86_64 2026-02-13T02:13:48.354 DEBUG:teuthology.orchestra.run.trial107:> sudo yum install -y kernel 2026-02-13T02:13:48.355 INFO:teuthology.orchestra.run.trial027.stdout:5.14.0-645.el9.x86_64 2026-02-13T02:13:48.355 INFO:teuthology.task.kernel:Running kernel on trial027: 5.14.0-645.el9.x86_64 2026-02-13T02:13:48.355 DEBUG:teuthology.orchestra.run.trial027:> sudo yum install -y kernel 2026-02-13T02:13:48.356 INFO:teuthology.orchestra.run.trial028.stdout:5.14.0-645.el9.x86_64 2026-02-13T02:13:48.356 INFO:teuthology.task.kernel:Running kernel on trial028: 5.14.0-645.el9.x86_64 2026-02-13T02:13:48.356 DEBUG:teuthology.orchestra.run.trial028:> sudo yum install -y kernel 2026-02-13T02:13:48.963 INFO:teuthology.orchestra.run.trial027.stdout:CentOS Stream 9 - BaseOS 37 MB/s | 8.9 MB 00:00 2026-02-13T02:13:48.999 INFO:teuthology.orchestra.run.trial107.stdout:CentOS Stream 9 - BaseOS 32 MB/s | 8.9 MB 00:00 2026-02-13T02:13:50.122 INFO:teuthology.orchestra.run.trial028.stdout:CentOS Stream 9 - BaseOS 6.3 MB/s | 8.9 MB 00:01 2026-02-13T02:13:52.171 INFO:teuthology.orchestra.run.trial027.stdout:CentOS Stream 9 - AppStream 10 MB/s | 27 MB 00:02 2026-02-13T02:13:53.347 INFO:teuthology.orchestra.run.trial028.stdout:CentOS Stream 9 - AppStream 9.9 MB/s | 27 MB 00:02 2026-02-13T02:13:55.706 INFO:teuthology.orchestra.run.trial027.stdout:CentOS Stream 9 - CRB 6.7 MB/s | 7.8 MB 00:01 2026-02-13T02:13:56.571 INFO:teuthology.orchestra.run.trial027.stdout:CentOS Stream 9 - Extras packages 139 kB/s | 20 kB 00:00 2026-02-13T02:13:58.057 INFO:teuthology.orchestra.run.trial027.stdout:Extra Packages for Enterprise Linux 14 MB/s | 20 MB 00:01 2026-02-13T02:14:01.294 INFO:teuthology.orchestra.run.trial027.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-13T02:14:02.352 INFO:teuthology.orchestra.run.trial027.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-13T02:14:02.372 INFO:teuthology.orchestra.run.trial027.stdout:Dependencies resolved. 2026-02-13T02:14:02.373 INFO:teuthology.orchestra.run.trial027.stdout:================================================================================ 2026-02-13T02:14:02.374 INFO:teuthology.orchestra.run.trial027.stdout: Package Architecture Version Repository Size 2026-02-13T02:14:02.374 INFO:teuthology.orchestra.run.trial027.stdout:================================================================================ 2026-02-13T02:14:02.374 INFO:teuthology.orchestra.run.trial027.stdout:Installing: 2026-02-13T02:14:02.374 INFO:teuthology.orchestra.run.trial027.stdout: kernel x86_64 5.14.0-677.el9 baseos 768 k 2026-02-13T02:14:02.374 INFO:teuthology.orchestra.run.trial027.stdout:Installing dependencies: 2026-02-13T02:14:02.374 INFO:teuthology.orchestra.run.trial027.stdout: kernel-core x86_64 5.14.0-677.el9 baseos 17 M 2026-02-13T02:14:02.374 INFO:teuthology.orchestra.run.trial027.stdout: kernel-modules x86_64 5.14.0-677.el9 baseos 40 M 2026-02-13T02:14:02.374 INFO:teuthology.orchestra.run.trial027.stdout: kernel-modules-core x86_64 5.14.0-677.el9 baseos 31 M 2026-02-13T02:14:02.374 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:14:02.374 INFO:teuthology.orchestra.run.trial027.stdout:Transaction Summary 2026-02-13T02:14:02.374 INFO:teuthology.orchestra.run.trial027.stdout:================================================================================ 2026-02-13T02:14:02.375 INFO:teuthology.orchestra.run.trial027.stdout:Install 4 Packages 2026-02-13T02:14:02.375 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:14:02.375 INFO:teuthology.orchestra.run.trial027.stdout:Total download size: 88 M 2026-02-13T02:14:02.375 INFO:teuthology.orchestra.run.trial027.stdout:Installed size: 135 M 2026-02-13T02:14:02.375 INFO:teuthology.orchestra.run.trial027.stdout:Downloading Packages: 2026-02-13T02:14:03.884 INFO:teuthology.orchestra.run.trial027.stdout:(1/4): kernel-5.14.0-677.el9.x86_64.rpm 529 kB/s | 768 kB 00:01 2026-02-13T02:14:04.571 INFO:teuthology.orchestra.run.trial028.stdout:CentOS Stream 9 - CRB 899 kB/s | 7.8 MB 00:08 2026-02-13T02:14:05.408 INFO:teuthology.orchestra.run.trial028.stdout:CentOS Stream 9 - Extras packages 195 kB/s | 20 kB 00:00 2026-02-13T02:14:06.679 INFO:teuthology.orchestra.run.trial028.stdout:Extra Packages for Enterprise Linux 17 MB/s | 20 MB 00:01 2026-02-13T02:14:07.470 INFO:teuthology.orchestra.run.trial107.stdout:CentOS Stream 9 - AppStream 1.5 MB/s | 27 MB 00:17 2026-02-13T02:14:08.099 INFO:teuthology.orchestra.run.trial027.stdout:(2/4): kernel-core-5.14.0-677.el9.x86_64.rpm 3.0 MB/s | 17 MB 00:05 2026-02-13T02:14:09.455 INFO:teuthology.orchestra.run.trial027.stdout:(3/4): kernel-modules-core-5.14.0-677.el9.x86_6 5.5 MB/s | 31 MB 00:05 2026-02-13T02:14:09.983 INFO:teuthology.orchestra.run.trial028.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-13T02:14:10.845 INFO:teuthology.orchestra.run.trial107.stdout:CentOS Stream 9 - CRB 7.5 MB/s | 7.8 MB 00:01 2026-02-13T02:14:11.043 INFO:teuthology.orchestra.run.trial028.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-13T02:14:11.063 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-13T02:14:11.064 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-13T02:14:11.064 INFO:teuthology.orchestra.run.trial028.stdout: Package Architecture Version Repository Size 2026-02-13T02:14:11.065 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-13T02:14:11.065 INFO:teuthology.orchestra.run.trial028.stdout:Installing: 2026-02-13T02:14:11.065 INFO:teuthology.orchestra.run.trial028.stdout: kernel x86_64 5.14.0-677.el9 baseos 768 k 2026-02-13T02:14:11.065 INFO:teuthology.orchestra.run.trial028.stdout:Installing dependencies: 2026-02-13T02:14:11.065 INFO:teuthology.orchestra.run.trial028.stdout: kernel-core x86_64 5.14.0-677.el9 baseos 17 M 2026-02-13T02:14:11.065 INFO:teuthology.orchestra.run.trial028.stdout: kernel-modules x86_64 5.14.0-677.el9 baseos 40 M 2026-02-13T02:14:11.065 INFO:teuthology.orchestra.run.trial028.stdout: kernel-modules-core x86_64 5.14.0-677.el9 baseos 31 M 2026-02-13T02:14:11.065 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:14:11.065 INFO:teuthology.orchestra.run.trial028.stdout:Transaction Summary 2026-02-13T02:14:11.065 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-13T02:14:11.065 INFO:teuthology.orchestra.run.trial028.stdout:Install 4 Packages 2026-02-13T02:14:11.066 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:14:11.066 INFO:teuthology.orchestra.run.trial028.stdout:Total download size: 88 M 2026-02-13T02:14:11.066 INFO:teuthology.orchestra.run.trial028.stdout:Installed size: 135 M 2026-02-13T02:14:11.066 INFO:teuthology.orchestra.run.trial028.stdout:Downloading Packages: 2026-02-13T02:14:11.710 INFO:teuthology.orchestra.run.trial107.stdout:CentOS Stream 9 - Extras packages 137 kB/s | 20 kB 00:00 2026-02-13T02:14:11.821 INFO:teuthology.orchestra.run.trial028.stdout:(1/4): kernel-5.14.0-677.el9.x86_64.rpm 1.1 MB/s | 768 kB 00:00 2026-02-13T02:14:12.129 INFO:teuthology.orchestra.run.trial107.stdout:Extra Packages for Enterprise Linux 59 MB/s | 20 MB 00:00 2026-02-13T02:14:12.404 INFO:teuthology.orchestra.run.trial028.stdout:(2/4): kernel-core-5.14.0-677.el9.x86_64.rpm 14 MB/s | 17 MB 00:01 2026-02-13T02:14:12.945 INFO:teuthology.orchestra.run.trial028.stdout:(3/4): kernel-modules-core-5.14.0-677.el9.x86_6 27 MB/s | 31 MB 00:01 2026-02-13T02:14:13.160 INFO:teuthology.orchestra.run.trial028.stdout:(4/4): kernel-modules-5.14.0-677.el9.x86_64.rpm 20 MB/s | 40 MB 00:01 2026-02-13T02:14:13.160 INFO:teuthology.orchestra.run.trial028.stdout:-------------------------------------------------------------------------------- 2026-02-13T02:14:13.160 INFO:teuthology.orchestra.run.trial028.stdout:Total 42 MB/s | 88 MB 00:02 2026-02-13T02:14:13.353 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction check 2026-02-13T02:14:13.568 INFO:teuthology.orchestra.run.trial028.stdout:Transaction check succeeded. 2026-02-13T02:14:13.568 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction test 2026-02-13T02:14:13.913 INFO:teuthology.orchestra.run.trial028.stdout:Transaction test succeeded. 2026-02-13T02:14:13.913 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction 2026-02-13T02:14:15.004 INFO:teuthology.orchestra.run.trial028.stdout: Preparing : 1/1 2026-02-13T02:14:15.287 INFO:teuthology.orchestra.run.trial028.stdout: Installing : kernel-modules-core-5.14.0-677.el9.x86_64 1/4 2026-02-13T02:14:15.306 INFO:teuthology.orchestra.run.trial028.stdout: Installing : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-13T02:14:15.331 INFO:teuthology.orchestra.run.trial107.stdout:lab-extras 2.7 MB/s | 50 kB 00:00 2026-02-13T02:14:16.391 INFO:teuthology.orchestra.run.trial107.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-13T02:14:16.411 INFO:teuthology.orchestra.run.trial107.stdout:Dependencies resolved. 2026-02-13T02:14:16.412 INFO:teuthology.orchestra.run.trial107.stdout:================================================================================ 2026-02-13T02:14:16.412 INFO:teuthology.orchestra.run.trial107.stdout: Package Architecture Version Repository Size 2026-02-13T02:14:16.413 INFO:teuthology.orchestra.run.trial107.stdout:================================================================================ 2026-02-13T02:14:16.413 INFO:teuthology.orchestra.run.trial107.stdout:Installing: 2026-02-13T02:14:16.413 INFO:teuthology.orchestra.run.trial107.stdout: kernel x86_64 5.14.0-677.el9 baseos 768 k 2026-02-13T02:14:16.413 INFO:teuthology.orchestra.run.trial107.stdout:Installing dependencies: 2026-02-13T02:14:16.413 INFO:teuthology.orchestra.run.trial107.stdout: kernel-core x86_64 5.14.0-677.el9 baseos 17 M 2026-02-13T02:14:16.413 INFO:teuthology.orchestra.run.trial107.stdout: kernel-modules x86_64 5.14.0-677.el9 baseos 40 M 2026-02-13T02:14:16.413 INFO:teuthology.orchestra.run.trial107.stdout: kernel-modules-core x86_64 5.14.0-677.el9 baseos 31 M 2026-02-13T02:14:16.413 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:14:16.413 INFO:teuthology.orchestra.run.trial107.stdout:Transaction Summary 2026-02-13T02:14:16.413 INFO:teuthology.orchestra.run.trial107.stdout:================================================================================ 2026-02-13T02:14:16.413 INFO:teuthology.orchestra.run.trial107.stdout:Install 4 Packages 2026-02-13T02:14:16.414 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:14:16.414 INFO:teuthology.orchestra.run.trial107.stdout:Total download size: 88 M 2026-02-13T02:14:16.414 INFO:teuthology.orchestra.run.trial107.stdout:Installed size: 135 M 2026-02-13T02:14:16.414 INFO:teuthology.orchestra.run.trial107.stdout:Downloading Packages: 2026-02-13T02:14:16.847 INFO:teuthology.orchestra.run.trial107.stdout:(1/4): kernel-5.14.0-677.el9.x86_64.rpm 5.8 MB/s | 768 kB 00:00 2026-02-13T02:14:16.992 INFO:teuthology.orchestra.run.trial107.stdout:(2/4): kernel-core-5.14.0-677.el9.x86_64.rpm 62 MB/s | 17 MB 00:00 2026-02-13T02:14:17.077 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-13T02:14:17.105 INFO:teuthology.orchestra.run.trial028.stdout: Installing : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-13T02:14:17.156 INFO:teuthology.orchestra.run.trial107.stdout:(3/4): kernel-modules-core-5.14.0-677.el9.x86_6 100 MB/s | 31 MB 00:00 2026-02-13T02:14:17.478 INFO:teuthology.orchestra.run.trial107.stdout:(4/4): kernel-modules-5.14.0-677.el9.x86_64.rpm 52 MB/s | 40 MB 00:00 2026-02-13T02:14:17.478 INFO:teuthology.orchestra.run.trial107.stdout:-------------------------------------------------------------------------------- 2026-02-13T02:14:17.478 INFO:teuthology.orchestra.run.trial107.stdout:Total 83 MB/s | 88 MB 00:01 2026-02-13T02:14:17.670 INFO:teuthology.orchestra.run.trial107.stdout:Running transaction check 2026-02-13T02:14:17.872 INFO:teuthology.orchestra.run.trial107.stdout:Transaction check succeeded. 2026-02-13T02:14:17.872 INFO:teuthology.orchestra.run.trial107.stdout:Running transaction test 2026-02-13T02:14:18.218 INFO:teuthology.orchestra.run.trial107.stdout:Transaction test succeeded. 2026-02-13T02:14:18.218 INFO:teuthology.orchestra.run.trial107.stdout:Running transaction 2026-02-13T02:14:19.312 INFO:teuthology.orchestra.run.trial107.stdout: Preparing : 1/1 2026-02-13T02:14:19.598 INFO:teuthology.orchestra.run.trial107.stdout: Installing : kernel-modules-core-5.14.0-677.el9.x86_64 1/4 2026-02-13T02:14:19.617 INFO:teuthology.orchestra.run.trial107.stdout: Installing : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-13T02:14:20.329 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-13T02:14:20.336 INFO:teuthology.orchestra.run.trial028.stdout: Installing : kernel-5.14.0-677.el9.x86_64 4/4 2026-02-13T02:14:21.386 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-13T02:14:21.415 INFO:teuthology.orchestra.run.trial107.stdout: Installing : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-13T02:14:23.438 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-13T02:14:24.502 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-13T02:14:24.508 INFO:teuthology.orchestra.run.trial107.stdout: Installing : kernel-5.14.0-677.el9.x86_64 4/4 2026-02-13T02:14:27.602 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-13T02:14:48.343 INFO:teuthology.orchestra.run.trial027.stdout:(4/4): kernel-modules-5.14.0-677.el9.x86_64.rpm 883 kB/s | 40 MB 00:45 2026-02-13T02:14:48.345 INFO:teuthology.orchestra.run.trial027.stdout:-------------------------------------------------------------------------------- 2026-02-13T02:14:48.345 INFO:teuthology.orchestra.run.trial027.stdout:Total 1.9 MB/s | 88 MB 00:45 2026-02-13T02:14:48.531 INFO:teuthology.orchestra.run.trial027.stdout:Running transaction check 2026-02-13T02:14:48.734 INFO:teuthology.orchestra.run.trial027.stdout:Transaction check succeeded. 2026-02-13T02:14:48.734 INFO:teuthology.orchestra.run.trial027.stdout:Running transaction test 2026-02-13T02:14:49.079 INFO:teuthology.orchestra.run.trial027.stdout:Transaction test succeeded. 2026-02-13T02:14:49.080 INFO:teuthology.orchestra.run.trial027.stdout:Running transaction 2026-02-13T02:14:50.171 INFO:teuthology.orchestra.run.trial027.stdout: Preparing : 1/1 2026-02-13T02:14:50.466 INFO:teuthology.orchestra.run.trial027.stdout: Installing : kernel-modules-core-5.14.0-677.el9.x86_64 1/4 2026-02-13T02:14:50.485 INFO:teuthology.orchestra.run.trial027.stdout: Installing : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-13T02:14:52.249 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-13T02:14:52.277 INFO:teuthology.orchestra.run.trial027.stdout: Installing : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-13T02:14:54.023 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 4/4 2026-02-13T02:14:54.046 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 4/4 2026-02-13T02:14:54.216 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: kernel-5.14.0-677.el9.x86_64 4/4 2026-02-13T02:14:54.216 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : kernel-5.14.0-677.el9.x86_64 1/4 2026-02-13T02:14:54.217 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-13T02:14:54.217 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-13T02:14:54.268 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-13T02:14:54.268 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:14:54.268 INFO:teuthology.orchestra.run.trial028.stdout:Installed: 2026-02-13T02:14:54.268 INFO:teuthology.orchestra.run.trial028.stdout: kernel-5.14.0-677.el9.x86_64 2026-02-13T02:14:54.268 INFO:teuthology.orchestra.run.trial028.stdout: kernel-core-5.14.0-677.el9.x86_64 2026-02-13T02:14:54.268 INFO:teuthology.orchestra.run.trial028.stdout: kernel-modules-5.14.0-677.el9.x86_64 2026-02-13T02:14:54.268 INFO:teuthology.orchestra.run.trial028.stdout: kernel-modules-core-5.14.0-677.el9.x86_64 2026-02-13T02:14:54.269 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:14:54.269 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-13T02:14:54.324 DEBUG:teuthology.orchestra.run.trial028:> rpm -q kernel | sort -rV | head -n 1 2026-02-13T02:14:54.350 INFO:teuthology.orchestra.run.trial028.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-13T02:14:54.351 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-13T02:14:54.351 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-645.el9.x86_64 Expected: 5.14.0-677.el9.x86_64 2026-02-13T02:14:54.351 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-13T02:14:54.351 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-13T02:14:54.352 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2026-02-13T02:14:54.352 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial028.front.sepia.ceph.com, path=None, version=distro) 2026-02-13T02:14:54.352 DEBUG:teuthology.orchestra.run.trial028:> rpm -q kernel | sort -rV | head -n 1 2026-02-13T02:14:54.414 INFO:teuthology.orchestra.run.trial028.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-13T02:14:54.415 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-13T02:14:54.415 DEBUG:teuthology.orchestra.run.trial028:> sudo rpm -qi grub2-tools 2026-02-13T02:14:54.485 INFO:teuthology.orchestra.run.trial028.stdout:Name : grub2-tools 2026-02-13T02:14:54.485 INFO:teuthology.orchestra.run.trial028.stdout:Epoch : 1 2026-02-13T02:14:54.485 INFO:teuthology.orchestra.run.trial028.stdout:Version : 2.06 2026-02-13T02:14:54.486 INFO:teuthology.orchestra.run.trial028.stdout:Release : 118.el9 2026-02-13T02:14:54.486 INFO:teuthology.orchestra.run.trial028.stdout:Architecture: x86_64 2026-02-13T02:14:54.486 INFO:teuthology.orchestra.run.trial028.stdout:Install Date: Thu 04 Dec 2025 11:54:21 PM UTC 2026-02-13T02:14:54.486 INFO:teuthology.orchestra.run.trial028.stdout:Group : Unspecified 2026-02-13T02:14:54.486 INFO:teuthology.orchestra.run.trial028.stdout:Size : 8393909 2026-02-13T02:14:54.486 INFO:teuthology.orchestra.run.trial028.stdout:License : GPLv3+ 2026-02-13T02:14:54.486 INFO:teuthology.orchestra.run.trial028.stdout:Signature : RSA/SHA256, Wed 05 Nov 2025 09:13:15 AM UTC, Key ID 05b555b38483c65d 2026-02-13T02:14:54.486 INFO:teuthology.orchestra.run.trial028.stdout:Source RPM : grub2-2.06-118.el9.src.rpm 2026-02-13T02:14:54.486 INFO:teuthology.orchestra.run.trial028.stdout:Build Date : Tue 04 Nov 2025 03:44:14 PM UTC 2026-02-13T02:14:54.486 INFO:teuthology.orchestra.run.trial028.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-13T02:14:54.486 INFO:teuthology.orchestra.run.trial028.stdout:Packager : builder@centos.org 2026-02-13T02:14:54.486 INFO:teuthology.orchestra.run.trial028.stdout:Vendor : CentOS 2026-02-13T02:14:54.486 INFO:teuthology.orchestra.run.trial028.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-13T02:14:54.487 INFO:teuthology.orchestra.run.trial028.stdout:Summary : Support tools for GRUB. 2026-02-13T02:14:54.487 INFO:teuthology.orchestra.run.trial028.stdout:Description : 2026-02-13T02:14:54.487 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:14:54.487 INFO:teuthology.orchestra.run.trial028.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-13T02:14:54.487 INFO:teuthology.orchestra.run.trial028.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-13T02:14:54.487 INFO:teuthology.orchestra.run.trial028.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-13T02:14:54.487 INFO:teuthology.orchestra.run.trial028.stdout:hardware devices. 2026-02-13T02:14:54.487 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:14:54.487 INFO:teuthology.orchestra.run.trial028.stdout:This subpackage provides tools for support of all platforms. 2026-02-13T02:14:54.488 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-13T02:14:54.488 INFO:teuthology.task.kernel:Updating grub on trial028 to boot 5.14.0-677.el9.x86_64 2026-02-13T02:14:54.489 DEBUG:teuthology.orchestra.run.trial028:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-13T02:14:54.552 INFO:teuthology.orchestra.run.trial028.stdout:/boot/loader/entries/2734456005d743dca7f3dc77c35a2d0d-0-rescue.conf 2026-02-13T02:14:54.553 DEBUG:teuthology.orchestra.run.trial028:> sudo command -v grubby && echo yes || echo no 2026-02-13T02:14:54.617 INFO:teuthology.orchestra.run.trial028.stdout:/sbin/grubby 2026-02-13T02:14:54.619 INFO:teuthology.orchestra.run.trial028.stdout:yes 2026-02-13T02:14:54.619 DEBUG:teuthology.orchestra.run.trial028:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-677.el9.x86_64" | head -n 1 2026-02-13T02:14:54.683 INFO:teuthology.orchestra.run.trial028.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64.conf 2026-02-13T02:14:54.684 DEBUG:teuthology.orchestra.run.trial028:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64 2026-02-13T02:14:54.756 DEBUG:teuthology.orchestra.run.trial028:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-13T02:14:54.967 INFO:teuthology.orchestra.run.trial028.stderr:Generating grub configuration file ... 2026-02-13T02:14:55.033 INFO:teuthology.orchestra.run.trial028.stderr:Found linux image: /boot/vmlinuz-5.14.0-677.el9.x86_64 2026-02-13T02:14:55.039 INFO:teuthology.orchestra.run.trial028.stderr:Found initrd image: /boot/initramfs-5.14.0-677.el9.x86_64.img 2026-02-13T02:14:55.293 INFO:teuthology.orchestra.run.trial028.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-13T02:14:55.299 INFO:teuthology.orchestra.run.trial028.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-13T02:14:55.301 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-13T02:14:55.307 INFO:teuthology.orchestra.run.trial027.stdout: Installing : kernel-5.14.0-677.el9.x86_64 4/4 2026-02-13T02:14:55.332 INFO:teuthology.orchestra.run.trial028.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-13T02:14:55.338 INFO:teuthology.orchestra.run.trial028.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-13T02:14:55.361 INFO:teuthology.orchestra.run.trial028.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-13T02:14:55.368 INFO:teuthology.orchestra.run.trial028.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-13T02:14:55.451 INFO:teuthology.orchestra.run.trial028.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-13T02:14:55.460 INFO:teuthology.orchestra.run.trial028.stderr:done 2026-02-13T02:14:55.462 DEBUG:teuthology.orchestra.run.trial028:> test -d /sys/firmware/efi 2026-02-13T02:14:55.478 DEBUG:teuthology.orchestra.run.trial028:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-13T02:14:55.545 INFO:teuthology.orchestra.run.trial028.stdout:/boot/efi/EFI/BOOT 2026-02-13T02:14:55.545 INFO:teuthology.orchestra.run.trial028.stdout:/boot/efi/EFI/centos 2026-02-13T02:14:55.547 DEBUG:teuthology.orchestra.run.trial028:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-13T02:14:55.613 INFO:teuthology.orchestra.run.trial028.stdout:no 2026-02-13T02:14:55.613 DEBUG:teuthology.orchestra.run.trial028:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-13T02:14:55.677 INFO:teuthology.orchestra.run.trial028.stdout:yes 2026-02-13T02:14:55.678 DEBUG:teuthology.orchestra.run.trial028:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-13T02:14:55.744 INFO:teuthology.task.kernel:Triggering safe hard reboot via SysRq on trial028 2026-02-13T02:14:55.745 DEBUG:teuthology.orchestra.run.trial028:> sudo bash -c 'echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger' 2026-02-13T02:14:57.977 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 4/4 2026-02-13T02:14:57.998 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 4/4 2026-02-13T02:14:58.062 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: kernel-5.14.0-677.el9.x86_64 4/4 2026-02-13T02:14:58.063 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : kernel-5.14.0-677.el9.x86_64 1/4 2026-02-13T02:14:58.063 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-13T02:14:58.063 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-13T02:14:58.115 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-13T02:14:58.115 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:14:58.115 INFO:teuthology.orchestra.run.trial107.stdout:Installed: 2026-02-13T02:14:58.115 INFO:teuthology.orchestra.run.trial107.stdout: kernel-5.14.0-677.el9.x86_64 2026-02-13T02:14:58.116 INFO:teuthology.orchestra.run.trial107.stdout: kernel-core-5.14.0-677.el9.x86_64 2026-02-13T02:14:58.116 INFO:teuthology.orchestra.run.trial107.stdout: kernel-modules-5.14.0-677.el9.x86_64 2026-02-13T02:14:58.116 INFO:teuthology.orchestra.run.trial107.stdout: kernel-modules-core-5.14.0-677.el9.x86_64 2026-02-13T02:14:58.116 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:14:58.116 INFO:teuthology.orchestra.run.trial107.stdout:Complete! 2026-02-13T02:14:58.174 DEBUG:teuthology.orchestra.run.trial107:> rpm -q kernel | sort -rV | head -n 1 2026-02-13T02:14:58.199 INFO:teuthology.orchestra.run.trial107.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-13T02:14:58.200 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-13T02:14:58.200 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-645.el9.x86_64 Expected: 5.14.0-677.el9.x86_64 2026-02-13T02:14:58.200 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-13T02:14:58.200 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-13T02:14:58.200 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2026-02-13T02:14:58.200 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial107.front.sepia.ceph.com, path=None, version=distro) 2026-02-13T02:14:58.201 DEBUG:teuthology.orchestra.run.trial107:> rpm -q kernel | sort -rV | head -n 1 2026-02-13T02:14:58.262 INFO:teuthology.orchestra.run.trial107.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-13T02:14:58.263 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-13T02:14:58.264 DEBUG:teuthology.orchestra.run.trial107:> sudo rpm -qi grub2-tools 2026-02-13T02:14:58.333 INFO:teuthology.orchestra.run.trial107.stdout:Name : grub2-tools 2026-02-13T02:14:58.333 INFO:teuthology.orchestra.run.trial107.stdout:Epoch : 1 2026-02-13T02:14:58.333 INFO:teuthology.orchestra.run.trial107.stdout:Version : 2.06 2026-02-13T02:14:58.333 INFO:teuthology.orchestra.run.trial107.stdout:Release : 118.el9 2026-02-13T02:14:58.334 INFO:teuthology.orchestra.run.trial107.stdout:Architecture: x86_64 2026-02-13T02:14:58.334 INFO:teuthology.orchestra.run.trial107.stdout:Install Date: Thu 04 Dec 2025 11:54:21 PM UTC 2026-02-13T02:14:58.334 INFO:teuthology.orchestra.run.trial107.stdout:Group : Unspecified 2026-02-13T02:14:58.334 INFO:teuthology.orchestra.run.trial107.stdout:Size : 8393909 2026-02-13T02:14:58.334 INFO:teuthology.orchestra.run.trial107.stdout:License : GPLv3+ 2026-02-13T02:14:58.334 INFO:teuthology.orchestra.run.trial107.stdout:Signature : RSA/SHA256, Wed 05 Nov 2025 09:13:15 AM UTC, Key ID 05b555b38483c65d 2026-02-13T02:14:58.334 INFO:teuthology.orchestra.run.trial107.stdout:Source RPM : grub2-2.06-118.el9.src.rpm 2026-02-13T02:14:58.334 INFO:teuthology.orchestra.run.trial107.stdout:Build Date : Tue 04 Nov 2025 03:44:14 PM UTC 2026-02-13T02:14:58.334 INFO:teuthology.orchestra.run.trial107.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-13T02:14:58.334 INFO:teuthology.orchestra.run.trial107.stdout:Packager : builder@centos.org 2026-02-13T02:14:58.335 INFO:teuthology.orchestra.run.trial107.stdout:Vendor : CentOS 2026-02-13T02:14:58.335 INFO:teuthology.orchestra.run.trial107.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-13T02:14:58.335 INFO:teuthology.orchestra.run.trial107.stdout:Summary : Support tools for GRUB. 2026-02-13T02:14:58.335 INFO:teuthology.orchestra.run.trial107.stdout:Description : 2026-02-13T02:14:58.335 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:14:58.335 INFO:teuthology.orchestra.run.trial107.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-13T02:14:58.335 INFO:teuthology.orchestra.run.trial107.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-13T02:14:58.335 INFO:teuthology.orchestra.run.trial107.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-13T02:14:58.335 INFO:teuthology.orchestra.run.trial107.stdout:hardware devices. 2026-02-13T02:14:58.335 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:14:58.336 INFO:teuthology.orchestra.run.trial107.stdout:This subpackage provides tools for support of all platforms. 2026-02-13T02:14:58.337 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-13T02:14:58.337 INFO:teuthology.task.kernel:Updating grub on trial107 to boot 5.14.0-677.el9.x86_64 2026-02-13T02:14:58.337 DEBUG:teuthology.orchestra.run.trial107:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-13T02:14:58.338 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-13T02:14:58.401 INFO:teuthology.orchestra.run.trial107.stdout:/boot/loader/entries/2734456005d743dca7f3dc77c35a2d0d-0-rescue.conf 2026-02-13T02:14:58.402 DEBUG:teuthology.orchestra.run.trial107:> sudo command -v grubby && echo yes || echo no 2026-02-13T02:14:58.468 INFO:teuthology.orchestra.run.trial107.stdout:/sbin/grubby 2026-02-13T02:14:58.469 INFO:teuthology.orchestra.run.trial107.stdout:yes 2026-02-13T02:14:58.469 DEBUG:teuthology.orchestra.run.trial107:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-677.el9.x86_64" | head -n 1 2026-02-13T02:14:58.532 INFO:teuthology.orchestra.run.trial107.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64.conf 2026-02-13T02:14:58.533 DEBUG:teuthology.orchestra.run.trial107:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64 2026-02-13T02:14:58.606 DEBUG:teuthology.orchestra.run.trial107:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-13T02:14:58.814 INFO:teuthology.orchestra.run.trial107.stderr:Generating grub configuration file ... 2026-02-13T02:14:58.876 INFO:teuthology.orchestra.run.trial107.stderr:Found linux image: /boot/vmlinuz-5.14.0-677.el9.x86_64 2026-02-13T02:14:58.882 INFO:teuthology.orchestra.run.trial107.stderr:Found initrd image: /boot/initramfs-5.14.0-677.el9.x86_64.img 2026-02-13T02:14:59.131 INFO:teuthology.orchestra.run.trial107.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-13T02:14:59.137 INFO:teuthology.orchestra.run.trial107.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-13T02:14:59.170 INFO:teuthology.orchestra.run.trial107.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-13T02:14:59.176 INFO:teuthology.orchestra.run.trial107.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-13T02:14:59.200 INFO:teuthology.orchestra.run.trial107.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-13T02:14:59.206 INFO:teuthology.orchestra.run.trial107.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-13T02:14:59.288 INFO:teuthology.orchestra.run.trial107.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-13T02:14:59.296 INFO:teuthology.orchestra.run.trial107.stderr:done 2026-02-13T02:14:59.298 DEBUG:teuthology.orchestra.run.trial107:> test -d /sys/firmware/efi 2026-02-13T02:14:59.314 DEBUG:teuthology.orchestra.run.trial107:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-13T02:14:59.379 INFO:teuthology.orchestra.run.trial107.stdout:/boot/efi/EFI/BOOT 2026-02-13T02:14:59.380 INFO:teuthology.orchestra.run.trial107.stdout:/boot/efi/EFI/centos 2026-02-13T02:14:59.381 DEBUG:teuthology.orchestra.run.trial107:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-13T02:14:59.445 INFO:teuthology.orchestra.run.trial107.stdout:no 2026-02-13T02:14:59.446 DEBUG:teuthology.orchestra.run.trial107:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-13T02:14:59.509 INFO:teuthology.orchestra.run.trial107.stdout:yes 2026-02-13T02:14:59.510 DEBUG:teuthology.orchestra.run.trial107:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-13T02:14:59.573 INFO:teuthology.task.kernel:Triggering safe hard reboot via SysRq on trial107 2026-02-13T02:14:59.573 DEBUG:teuthology.orchestra.run.trial107:> sudo bash -c 'echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger' 2026-02-13T02:15:25.787 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2026-02-13T02:15:25.788 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial028.front.sepia.ceph.com' 2026-02-13T02:15:25.788 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T02:15:28.852 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 4/4 2026-02-13T02:15:28.873 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 4/4 2026-02-13T02:15:28.937 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: kernel-5.14.0-677.el9.x86_64 4/4 2026-02-13T02:15:28.938 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : kernel-5.14.0-677.el9.x86_64 1/4 2026-02-13T02:15:28.938 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-13T02:15:28.938 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-13T02:15:28.989 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-13T02:15:28.990 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:15:28.990 INFO:teuthology.orchestra.run.trial027.stdout:Installed: 2026-02-13T02:15:28.990 INFO:teuthology.orchestra.run.trial027.stdout: kernel-5.14.0-677.el9.x86_64 2026-02-13T02:15:28.990 INFO:teuthology.orchestra.run.trial027.stdout: kernel-core-5.14.0-677.el9.x86_64 2026-02-13T02:15:28.990 INFO:teuthology.orchestra.run.trial027.stdout: kernel-modules-5.14.0-677.el9.x86_64 2026-02-13T02:15:28.990 INFO:teuthology.orchestra.run.trial027.stdout: kernel-modules-core-5.14.0-677.el9.x86_64 2026-02-13T02:15:28.990 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:15:28.990 INFO:teuthology.orchestra.run.trial027.stdout:Complete! 2026-02-13T02:15:29.046 DEBUG:teuthology.orchestra.run.trial027:> rpm -q kernel | sort -rV | head -n 1 2026-02-13T02:15:29.072 INFO:teuthology.orchestra.run.trial027.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-13T02:15:29.073 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-13T02:15:29.073 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-645.el9.x86_64 Expected: 5.14.0-677.el9.x86_64 2026-02-13T02:15:29.073 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-13T02:15:29.073 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-13T02:15:29.074 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-13T02:15:29.074 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial027.front.sepia.ceph.com, path=None, version=distro) 2026-02-13T02:15:29.074 DEBUG:teuthology.orchestra.run.trial027:> rpm -q kernel | sort -rV | head -n 1 2026-02-13T02:15:29.136 INFO:teuthology.orchestra.run.trial027.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-13T02:15:29.137 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-13T02:15:29.137 DEBUG:teuthology.orchestra.run.trial027:> sudo rpm -qi grub2-tools 2026-02-13T02:15:29.207 INFO:teuthology.orchestra.run.trial027.stdout:Name : grub2-tools 2026-02-13T02:15:29.207 INFO:teuthology.orchestra.run.trial027.stdout:Epoch : 1 2026-02-13T02:15:29.207 INFO:teuthology.orchestra.run.trial027.stdout:Version : 2.06 2026-02-13T02:15:29.207 INFO:teuthology.orchestra.run.trial027.stdout:Release : 118.el9 2026-02-13T02:15:29.207 INFO:teuthology.orchestra.run.trial027.stdout:Architecture: x86_64 2026-02-13T02:15:29.208 INFO:teuthology.orchestra.run.trial027.stdout:Install Date: Thu 04 Dec 2025 11:54:21 PM UTC 2026-02-13T02:15:29.208 INFO:teuthology.orchestra.run.trial027.stdout:Group : Unspecified 2026-02-13T02:15:29.208 INFO:teuthology.orchestra.run.trial027.stdout:Size : 8393909 2026-02-13T02:15:29.208 INFO:teuthology.orchestra.run.trial027.stdout:License : GPLv3+ 2026-02-13T02:15:29.208 INFO:teuthology.orchestra.run.trial027.stdout:Signature : RSA/SHA256, Wed 05 Nov 2025 09:13:15 AM UTC, Key ID 05b555b38483c65d 2026-02-13T02:15:29.208 INFO:teuthology.orchestra.run.trial027.stdout:Source RPM : grub2-2.06-118.el9.src.rpm 2026-02-13T02:15:29.208 INFO:teuthology.orchestra.run.trial027.stdout:Build Date : Tue 04 Nov 2025 03:44:14 PM UTC 2026-02-13T02:15:29.208 INFO:teuthology.orchestra.run.trial027.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-13T02:15:29.208 INFO:teuthology.orchestra.run.trial027.stdout:Packager : builder@centos.org 2026-02-13T02:15:29.208 INFO:teuthology.orchestra.run.trial027.stdout:Vendor : CentOS 2026-02-13T02:15:29.208 INFO:teuthology.orchestra.run.trial027.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-13T02:15:29.208 INFO:teuthology.orchestra.run.trial027.stdout:Summary : Support tools for GRUB. 2026-02-13T02:15:29.209 INFO:teuthology.orchestra.run.trial027.stdout:Description : 2026-02-13T02:15:29.209 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:15:29.209 INFO:teuthology.orchestra.run.trial027.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-13T02:15:29.209 INFO:teuthology.orchestra.run.trial027.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-13T02:15:29.209 INFO:teuthology.orchestra.run.trial027.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-13T02:15:29.209 INFO:teuthology.orchestra.run.trial027.stdout:hardware devices. 2026-02-13T02:15:29.209 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:15:29.209 INFO:teuthology.orchestra.run.trial027.stdout:This subpackage provides tools for support of all platforms. 2026-02-13T02:15:29.210 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-13T02:15:29.210 INFO:teuthology.task.kernel:Updating grub on trial027 to boot 5.14.0-677.el9.x86_64 2026-02-13T02:15:29.211 DEBUG:teuthology.orchestra.run.trial027:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-13T02:15:29.275 INFO:teuthology.orchestra.run.trial027.stdout:/boot/loader/entries/2734456005d743dca7f3dc77c35a2d0d-0-rescue.conf 2026-02-13T02:15:29.276 DEBUG:teuthology.orchestra.run.trial027:> sudo command -v grubby && echo yes || echo no 2026-02-13T02:15:29.342 INFO:teuthology.orchestra.run.trial027.stdout:/sbin/grubby 2026-02-13T02:15:29.344 INFO:teuthology.orchestra.run.trial027.stdout:yes 2026-02-13T02:15:29.344 DEBUG:teuthology.orchestra.run.trial027:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-677.el9.x86_64" | head -n 1 2026-02-13T02:15:29.408 INFO:teuthology.orchestra.run.trial027.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64.conf 2026-02-13T02:15:29.409 DEBUG:teuthology.orchestra.run.trial027:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64 2026-02-13T02:15:29.482 DEBUG:teuthology.orchestra.run.trial027:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-13T02:15:29.615 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2026-02-13T02:15:29.616 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial107.front.sepia.ceph.com' 2026-02-13T02:15:29.617 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T02:15:29.692 INFO:teuthology.orchestra.run.trial027.stderr:Generating grub configuration file ... 2026-02-13T02:15:29.756 INFO:teuthology.orchestra.run.trial027.stderr:Found linux image: /boot/vmlinuz-5.14.0-677.el9.x86_64 2026-02-13T02:15:29.762 INFO:teuthology.orchestra.run.trial027.stderr:Found initrd image: /boot/initramfs-5.14.0-677.el9.x86_64.img 2026-02-13T02:15:30.014 INFO:teuthology.orchestra.run.trial027.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-13T02:15:30.021 INFO:teuthology.orchestra.run.trial027.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-13T02:15:30.053 INFO:teuthology.orchestra.run.trial027.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-13T02:15:30.060 INFO:teuthology.orchestra.run.trial027.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-13T02:15:30.083 INFO:teuthology.orchestra.run.trial027.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-13T02:15:30.090 INFO:teuthology.orchestra.run.trial027.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-13T02:15:30.173 INFO:teuthology.orchestra.run.trial027.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-13T02:15:30.182 INFO:teuthology.orchestra.run.trial027.stderr:done 2026-02-13T02:15:30.184 DEBUG:teuthology.orchestra.run.trial027:> test -d /sys/firmware/efi 2026-02-13T02:15:30.200 DEBUG:teuthology.orchestra.run.trial027:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-13T02:15:30.265 INFO:teuthology.orchestra.run.trial027.stdout:/boot/efi/EFI/BOOT 2026-02-13T02:15:30.265 INFO:teuthology.orchestra.run.trial027.stdout:/boot/efi/EFI/centos 2026-02-13T02:15:30.267 DEBUG:teuthology.orchestra.run.trial027:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-13T02:15:30.334 INFO:teuthology.orchestra.run.trial027.stdout:no 2026-02-13T02:15:30.334 DEBUG:teuthology.orchestra.run.trial027:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-13T02:15:30.398 INFO:teuthology.orchestra.run.trial027.stdout:yes 2026-02-13T02:15:30.398 DEBUG:teuthology.orchestra.run.trial027:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-13T02:15:30.462 INFO:teuthology.task.kernel:Triggering safe hard reboot via SysRq on trial027 2026-02-13T02:15:30.462 DEBUG:teuthology.orchestra.run.trial027:> sudo bash -c 'echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger' 2026-02-13T02:15:40.061 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.28 2026-02-13T02:15:43.901 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.107 2026-02-13T02:15:49.061 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial028.front.sepia.ceph.com' 2026-02-13T02:15:49.062 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T02:15:52.902 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial107.front.sepia.ceph.com' 2026-02-13T02:15:52.903 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T02:16:00.504 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-13T02:16:00.505 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial027.front.sepia.ceph.com' 2026-02-13T02:16:00.505 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T02:16:14.813 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.27 2026-02-13T02:16:23.822 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial027.front.sepia.ceph.com' 2026-02-13T02:16:23.822 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T02:16:25.117 DEBUG:teuthology.orchestra.run.trial028:> true 2026-02-13T02:16:25.278 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial028.front.sepia.ceph.com' 2026-02-13T02:16:25.278 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-677.el9.x86_64"... 2026-02-13T02:16:25.278 DEBUG:teuthology.orchestra.run.trial028:> uname -r 2026-02-13T02:16:25.333 INFO:teuthology.orchestra.run.trial028.stdout:5.14.0-677.el9.x86_64 2026-02-13T02:16:25.334 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-677.el9.x86_64 vs 5.14.0-677.el9.x86_64 2026-02-13T02:16:25.334 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-13T02:16:25.334 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-13T02:16:26.335 DEBUG:teuthology.parallel:result is None 2026-02-13T02:16:28.729 ERROR:teuthology.orchestra.connection:Error authenticating with trial107.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_a2c505dae37458c5537fa44f1320bd04eef25bb1/teuthology/orchestra/connection.py", line 111, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_a2c505dae37458c5537fa44f1320bd04eef25bb1/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect self._auth( File "/home/teuthworker/src/git.ceph.com_teuthology_a2c505dae37458c5537fa44f1320bd04eef25bb1/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth raise saved_exception File "/home/teuthworker/src/git.ceph.com_teuthology_a2c505dae37458c5537fa44f1320bd04eef25bb1/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth self._transport.auth_publickey(username, key) File "/home/teuthworker/src/git.ceph.com_teuthology_a2c505dae37458c5537fa44f1320bd04eef25bb1/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey raise SSHException("No existing session") paramiko.ssh_exception.SSHException: No existing session 2026-02-13T02:16:32.844 DEBUG:teuthology.orchestra.run.trial107:> true 2026-02-13T02:16:33.038 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial107.front.sepia.ceph.com' 2026-02-13T02:16:33.038 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-677.el9.x86_64"... 2026-02-13T02:16:33.038 DEBUG:teuthology.orchestra.run.trial107:> uname -r 2026-02-13T02:16:33.094 INFO:teuthology.orchestra.run.trial107.stdout:5.14.0-677.el9.x86_64 2026-02-13T02:16:33.094 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-677.el9.x86_64 vs 5.14.0-677.el9.x86_64 2026-02-13T02:16:33.094 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-13T02:16:33.094 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-13T02:16:34.095 DEBUG:teuthology.parallel:result is None 2026-02-13T02:16:59.932 DEBUG:teuthology.orchestra.run.trial027:> true 2026-02-13T02:17:00.091 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial027.front.sepia.ceph.com' 2026-02-13T02:17:00.091 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-677.el9.x86_64"... 2026-02-13T02:17:00.091 DEBUG:teuthology.orchestra.run.trial027:> uname -r 2026-02-13T02:17:00.148 INFO:teuthology.orchestra.run.trial027.stdout:5.14.0-677.el9.x86_64 2026-02-13T02:17:00.149 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-677.el9.x86_64 vs 5.14.0-677.el9.x86_64 2026-02-13T02:17:00.149 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-13T02:17:00.149 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-13T02:17:01.150 DEBUG:teuthology.parallel:result is None 2026-02-13T02:17:01.151 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-13T02:17:01.165 INFO:teuthology.task.internal:Creating test directory... 2026-02-13T02:17:01.165 DEBUG:teuthology.orchestra.run.trial027:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-13T02:17:01.168 DEBUG:teuthology.orchestra.run.trial028:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-13T02:17:01.171 DEBUG:teuthology.orchestra.run.trial107:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-13T02:17:01.191 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-13T02:17:01.203 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-13T02:17:01.216 INFO:teuthology.task.internal:Creating archive directory... 2026-02-13T02:17:01.216 DEBUG:teuthology.orchestra.run.trial027:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-13T02:17:01.225 DEBUG:teuthology.orchestra.run.trial028:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-13T02:17:01.230 DEBUG:teuthology.orchestra.run.trial107:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-13T02:17:01.259 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-13T02:17:01.272 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-13T02:17:01.273 DEBUG:teuthology.orchestra.run.trial027:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-13T02:17:01.282 DEBUG:teuthology.orchestra.run.trial028:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-13T02:17:01.288 DEBUG:teuthology.orchestra.run.trial107:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-13T02:17:01.311 INFO:teuthology.orchestra.run.trial027.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T02:17:01.314 INFO:teuthology.orchestra.run.trial028.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T02:17:01.319 INFO:teuthology.orchestra.run.trial107.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T02:17:01.320 INFO:teuthology.orchestra.run.trial027.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T02:17:01.324 INFO:teuthology.orchestra.run.trial028.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T02:17:01.328 INFO:teuthology.orchestra.run.trial107.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-13T02:17:01.330 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-13T02:17:01.346 INFO:teuthology.task.internal:Configuring sudo... 2026-02-13T02:17:01.346 DEBUG:teuthology.orchestra.run.trial027:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-13T02:17:01.363 DEBUG:teuthology.orchestra.run.trial028:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-13T02:17:01.368 DEBUG:teuthology.orchestra.run.trial107:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-13T02:17:01.394 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-13T02:17:01.411 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-13T02:17:01.411 DEBUG:teuthology.orchestra.run.trial027:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-13T02:17:01.427 DEBUG:teuthology.orchestra.run.trial028:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-13T02:17:01.433 DEBUG:teuthology.orchestra.run.trial107:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-13T02:17:01.449 DEBUG:teuthology.orchestra.run.trial027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-13T02:17:01.599 DEBUG:teuthology.orchestra.run.trial027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-13T02:17:01.624 DEBUG:teuthology.orchestra.run.trial027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-13T02:17:01.768 DEBUG:teuthology.orchestra.run.trial027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-13T02:17:01.792 DEBUG:teuthology.orchestra.run.trial027:> set -ex 2026-02-13T02:17:01.792 DEBUG:teuthology.orchestra.run.trial027:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-13T02:17:01.855 DEBUG:teuthology.orchestra.run.trial028:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-13T02:17:01.969 DEBUG:teuthology.orchestra.run.trial028:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-13T02:17:01.994 DEBUG:teuthology.orchestra.run.trial028:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-13T02:17:02.147 DEBUG:teuthology.orchestra.run.trial028:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-13T02:17:02.171 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-13T02:17:02.171 DEBUG:teuthology.orchestra.run.trial028:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-13T02:17:02.234 DEBUG:teuthology.orchestra.run.trial107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-13T02:17:02.349 DEBUG:teuthology.orchestra.run.trial107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-13T02:17:02.373 DEBUG:teuthology.orchestra.run.trial107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-13T02:17:02.521 DEBUG:teuthology.orchestra.run.trial107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-13T02:17:02.546 DEBUG:teuthology.orchestra.run.trial107:> set -ex 2026-02-13T02:17:02.546 DEBUG:teuthology.orchestra.run.trial107:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-13T02:17:02.609 DEBUG:teuthology.orchestra.run.trial027:> sudo service rsyslog restart 2026-02-13T02:17:02.611 DEBUG:teuthology.orchestra.run.trial028:> sudo service rsyslog restart 2026-02-13T02:17:02.613 DEBUG:teuthology.orchestra.run.trial107:> sudo service rsyslog restart 2026-02-13T02:17:02.636 INFO:teuthology.orchestra.run.trial027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-13T02:17:02.639 INFO:teuthology.orchestra.run.trial028.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-13T02:17:02.675 INFO:teuthology.orchestra.run.trial107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-13T02:17:03.153 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-13T02:17:03.167 INFO:teuthology.task.internal:Starting timer... 2026-02-13T02:17:03.167 INFO:teuthology.run_tasks:Running task pcp... 2026-02-13T02:17:03.189 INFO:teuthology.run_tasks:Running task selinux... 2026-02-13T02:17:03.207 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2026-02-13T02:17:03.207 DEBUG:teuthology.orchestra.run.trial027:> sudo service auditd rotate 2026-02-13T02:17:03.237 INFO:teuthology.orchestra.run.trial027.stdout:Rotating logs: 2026-02-13T02:17:03.239 DEBUG:teuthology.orchestra.run.trial028:> sudo service auditd rotate 2026-02-13T02:17:03.269 INFO:teuthology.orchestra.run.trial028.stdout:Rotating logs: 2026-02-13T02:17:03.270 DEBUG:teuthology.orchestra.run.trial107:> sudo service auditd rotate 2026-02-13T02:17:03.299 INFO:teuthology.orchestra.run.trial107.stdout:Rotating logs: 2026-02-13T02:17:03.300 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-13T02:17:03.301 DEBUG:teuthology.orchestra.run.trial027:> /usr/sbin/getenforce 2026-02-13T02:17:03.317 INFO:teuthology.orchestra.run.trial027.stdout:Permissive 2026-02-13T02:17:03.317 DEBUG:teuthology.orchestra.run.trial028:> /usr/sbin/getenforce 2026-02-13T02:17:03.333 INFO:teuthology.orchestra.run.trial028.stdout:Permissive 2026-02-13T02:17:03.334 DEBUG:teuthology.orchestra.run.trial107:> /usr/sbin/getenforce 2026-02-13T02:17:03.357 INFO:teuthology.orchestra.run.trial107.stdout:Permissive 2026-02-13T02:17:03.358 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial027.front.sepia.ceph.com': 'permissive', 'ubuntu@trial028.front.sepia.ceph.com': 'permissive', 'ubuntu@trial107.front.sepia.ceph.com': 'permissive'} 2026-02-13T02:17:03.358 DEBUG:teuthology.orchestra.run.trial027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-13T02:17:03.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T02:17:03.383 DEBUG:teuthology.orchestra.run.trial028:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-13T02:17:03.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T02:17:03.409 DEBUG:teuthology.orchestra.run.trial107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-13T02:17:03.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T02:17:03.434 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-13T02:17:03.434 DEBUG:teuthology.orchestra.run.trial027:> sudo /usr/sbin/setenforce permissive 2026-02-13T02:17:03.459 DEBUG:teuthology.orchestra.run.trial028:> sudo /usr/sbin/setenforce permissive 2026-02-13T02:17:03.484 DEBUG:teuthology.orchestra.run.trial107:> sudo /usr/sbin/setenforce permissive 2026-02-13T02:17:03.508 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-13T02:17:03.526 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-13T02:17:03.537 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-13T02:17:03.538 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial027.front.sepia.ceph.com,trial028.front.sepia.ceph.com,trial107.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-13T02:18:16.053 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial027.front.sepia.ceph.com'), Remote(name='ubuntu@trial028.front.sepia.ceph.com'), Remote(name='ubuntu@trial107.front.sepia.ceph.com')] 2026-02-13T02:18:16.055 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial027.front.sepia.ceph.com' 2026-02-13T02:18:16.057 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T02:18:16.166 DEBUG:teuthology.orchestra.run.trial027:> true 2026-02-13T02:18:16.248 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial027.front.sepia.ceph.com' 2026-02-13T02:18:16.249 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial028.front.sepia.ceph.com' 2026-02-13T02:18:16.249 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T02:18:16.334 DEBUG:teuthology.orchestra.run.trial028:> true 2026-02-13T02:18:16.408 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial028.front.sepia.ceph.com' 2026-02-13T02:18:16.408 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial107.front.sepia.ceph.com' 2026-02-13T02:18:16.409 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-13T02:18:16.500 DEBUG:teuthology.orchestra.run.trial107:> true 2026-02-13T02:18:16.575 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial107.front.sepia.ceph.com' 2026-02-13T02:18:16.575 INFO:teuthology.run_tasks:Running task clock... 2026-02-13T02:18:16.595 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-13T02:18:16.595 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-13T02:18:16.596 DEBUG:teuthology.orchestra.run.trial027:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-13T02:18:16.598 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-13T02:18:16.598 DEBUG:teuthology.orchestra.run.trial028:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-13T02:18:16.601 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-13T02:18:16.601 DEBUG:teuthology.orchestra.run.trial107:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-13T02:18:16.624 INFO:teuthology.orchestra.run.trial027.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-13T02:18:16.627 INFO:teuthology.orchestra.run.trial028.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-13T02:18:16.635 INFO:teuthology.orchestra.run.trial027.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-13T02:18:16.638 INFO:teuthology.orchestra.run.trial028.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-13T02:18:16.644 INFO:teuthology.orchestra.run.trial107.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-13T02:18:16.655 INFO:teuthology.orchestra.run.trial107.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-13T02:18:16.656 INFO:teuthology.orchestra.run.trial027.stderr:sudo: ntpd: command not found 2026-02-13T02:18:16.660 INFO:teuthology.orchestra.run.trial028.stderr:sudo: ntpd: command not found 2026-02-13T02:18:16.665 INFO:teuthology.orchestra.run.trial027.stdout:506 Cannot talk to daemon 2026-02-13T02:18:16.669 INFO:teuthology.orchestra.run.trial028.stdout:506 Cannot talk to daemon 2026-02-13T02:18:16.674 INFO:teuthology.orchestra.run.trial107.stderr:sudo: ntpd: command not found 2026-02-13T02:18:16.676 INFO:teuthology.orchestra.run.trial027.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-13T02:18:16.681 INFO:teuthology.orchestra.run.trial028.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-13T02:18:16.684 INFO:teuthology.orchestra.run.trial107.stdout:506 Cannot talk to daemon 2026-02-13T02:18:16.687 INFO:teuthology.orchestra.run.trial027.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-13T02:18:16.693 INFO:teuthology.orchestra.run.trial028.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-13T02:18:16.695 INFO:teuthology.orchestra.run.trial107.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-13T02:18:16.706 INFO:teuthology.orchestra.run.trial107.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-13T02:18:16.739 INFO:teuthology.orchestra.run.trial027.stderr:bash: line 1: ntpq: command not found 2026-02-13T02:18:16.749 INFO:teuthology.orchestra.run.trial028.stderr:bash: line 1: ntpq: command not found 2026-02-13T02:18:16.753 INFO:teuthology.orchestra.run.trial107.stderr:bash: line 1: ntpq: command not found 2026-02-13T02:18:16.755 INFO:teuthology.orchestra.run.trial027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-13T02:18:16.755 INFO:teuthology.orchestra.run.trial027.stdout:=============================================================================== 2026-02-13T02:18:16.756 INFO:teuthology.orchestra.run.trial027.stdout:^? 23.186.168.131 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-13T02:18:16.756 INFO:teuthology.orchestra.run.trial027.stdout:^? nyc2.us.ntp.li 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-13T02:18:16.756 INFO:teuthology.orchestra.run.trial027.stdout:^? 172-234-44-141.ip.linode> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-13T02:18:16.756 INFO:teuthology.orchestra.run.trial027.stdout:^? chi2.us.ntp.li 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-13T02:18:16.781 INFO:teuthology.orchestra.run.trial107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-13T02:18:16.781 INFO:teuthology.orchestra.run.trial107.stdout:=============================================================================== 2026-02-13T02:18:16.782 INFO:teuthology.orchestra.run.trial107.stdout:^? 66.42.71.197.vultruserco> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-13T02:18:16.782 INFO:teuthology.orchestra.run.trial107.stdout:^? 23.186.168.133 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-13T02:18:16.782 INFO:teuthology.orchestra.run.trial107.stdout:^? time.walb.tech 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-13T02:18:16.782 INFO:teuthology.orchestra.run.trial107.stdout:^? ntp.maxhost.io 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-13T02:18:16.785 INFO:teuthology.orchestra.run.trial028.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-13T02:18:16.785 INFO:teuthology.orchestra.run.trial028.stdout:=============================================================================== 2026-02-13T02:18:16.785 INFO:teuthology.orchestra.run.trial028.stdout:^? 172-232-15-202.ip.linode> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-13T02:18:16.785 INFO:teuthology.orchestra.run.trial028.stdout:^? edge-anycast.ntp00.hosti> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-13T02:18:16.785 INFO:teuthology.orchestra.run.trial028.stdout:^? 69-10-208-170.rainiercon> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-13T02:18:16.785 INFO:teuthology.orchestra.run.trial028.stdout:^? 172-104-28-175.ip.linode> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-13T02:18:16.786 INFO:teuthology.run_tasks:Running task pexec... 2026-02-13T02:18:16.804 INFO:teuthology.task.pexec:Executing custom commands... 2026-02-13T02:18:16.805 DEBUG:teuthology.orchestra.run.trial027:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-13T02:18:16.805 DEBUG:teuthology.orchestra.run.trial028:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-13T02:18:16.806 DEBUG:teuthology.orchestra.run.trial107:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-13T02:18:16.809 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial027.front.sepia.ceph.com 2026-02-13T02:18:16.809 INFO:teuthology.task.pexec:sudo dnf remove nvme-cli -y 2026-02-13T02:18:16.809 INFO:teuthology.task.pexec:sudo dnf install nvmetcli nvme-cli -y 2026-02-13T02:18:16.827 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial107.front.sepia.ceph.com 2026-02-13T02:18:16.827 INFO:teuthology.task.pexec:sudo dnf remove nvme-cli -y 2026-02-13T02:18:16.827 INFO:teuthology.task.pexec:sudo dnf install nvmetcli nvme-cli -y 2026-02-13T02:18:16.830 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial028.front.sepia.ceph.com 2026-02-13T02:18:16.830 INFO:teuthology.task.pexec:sudo dnf remove nvme-cli -y 2026-02-13T02:18:16.830 INFO:teuthology.task.pexec:sudo dnf install nvmetcli nvme-cli -y 2026-02-13T02:18:16.982 INFO:teuthology.orchestra.run.trial027.stdout:No match for argument: nvme-cli 2026-02-13T02:18:16.983 INFO:teuthology.orchestra.run.trial027.stderr:No packages marked for removal. 2026-02-13T02:18:16.985 INFO:teuthology.orchestra.run.trial027.stdout:Dependencies resolved. 2026-02-13T02:18:16.986 INFO:teuthology.orchestra.run.trial027.stdout:Nothing to do. 2026-02-13T02:18:16.986 INFO:teuthology.orchestra.run.trial027.stdout:Complete! 2026-02-13T02:18:17.000 INFO:teuthology.orchestra.run.trial107.stdout:No match for argument: nvme-cli 2026-02-13T02:18:17.000 INFO:teuthology.orchestra.run.trial107.stderr:No packages marked for removal. 2026-02-13T02:18:17.001 INFO:teuthology.orchestra.run.trial028.stdout:No match for argument: nvme-cli 2026-02-13T02:18:17.002 INFO:teuthology.orchestra.run.trial028.stderr:No packages marked for removal. 2026-02-13T02:18:17.002 INFO:teuthology.orchestra.run.trial107.stdout:Dependencies resolved. 2026-02-13T02:18:17.003 INFO:teuthology.orchestra.run.trial107.stdout:Nothing to do. 2026-02-13T02:18:17.003 INFO:teuthology.orchestra.run.trial107.stdout:Complete! 2026-02-13T02:18:17.004 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-13T02:18:17.005 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-13T02:18:17.005 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-13T02:18:17.282 INFO:teuthology.orchestra.run.trial027.stdout:Last metadata expiration check: 0:00:33 ago on Fri 13 Feb 2026 02:17:44 AM UTC. 2026-02-13T02:18:17.296 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:00:33 ago on Fri 13 Feb 2026 02:17:44 AM UTC. 2026-02-13T02:18:17.299 INFO:teuthology.orchestra.run.trial107.stdout:Last metadata expiration check: 0:00:33 ago on Fri 13 Feb 2026 02:17:44 AM UTC. 2026-02-13T02:18:17.364 INFO:teuthology.orchestra.run.trial027.stdout:Dependencies resolved. 2026-02-13T02:18:17.365 INFO:teuthology.orchestra.run.trial027.stdout:================================================================================ 2026-02-13T02:18:17.365 INFO:teuthology.orchestra.run.trial027.stdout: Package Architecture Version Repository Size 2026-02-13T02:18:17.365 INFO:teuthology.orchestra.run.trial027.stdout:================================================================================ 2026-02-13T02:18:17.365 INFO:teuthology.orchestra.run.trial027.stdout:Installing: 2026-02-13T02:18:17.365 INFO:teuthology.orchestra.run.trial027.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-13T02:18:17.365 INFO:teuthology.orchestra.run.trial027.stdout: nvmetcli noarch 0.8-3.el9 baseos 44 k 2026-02-13T02:18:17.365 INFO:teuthology.orchestra.run.trial027.stdout:Upgrading: 2026-02-13T02:18:17.366 INFO:teuthology.orchestra.run.trial027.stdout: libnvme x86_64 1.16.1-2.el9 baseos 116 k 2026-02-13T02:18:17.366 INFO:teuthology.orchestra.run.trial027.stdout:Installing dependencies: 2026-02-13T02:18:17.366 INFO:teuthology.orchestra.run.trial027.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2026-02-13T02:18:17.366 INFO:teuthology.orchestra.run.trial027.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2026-02-13T02:18:17.366 INFO:teuthology.orchestra.run.trial027.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-13T02:18:17.366 INFO:teuthology.orchestra.run.trial027.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2026-02-13T02:18:17.366 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:18:17.366 INFO:teuthology.orchestra.run.trial027.stdout:Transaction Summary 2026-02-13T02:18:17.366 INFO:teuthology.orchestra.run.trial027.stdout:================================================================================ 2026-02-13T02:18:17.366 INFO:teuthology.orchestra.run.trial027.stdout:Install 6 Packages 2026-02-13T02:18:17.366 INFO:teuthology.orchestra.run.trial027.stdout:Upgrade 1 Package 2026-02-13T02:18:17.366 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:18:17.367 INFO:teuthology.orchestra.run.trial027.stdout:Total download size: 2.4 M 2026-02-13T02:18:17.367 INFO:teuthology.orchestra.run.trial027.stdout:Downloading Packages: 2026-02-13T02:18:17.377 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-13T02:18:17.378 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-13T02:18:17.378 INFO:teuthology.orchestra.run.trial028.stdout: Package Architecture Version Repository Size 2026-02-13T02:18:17.378 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-13T02:18:17.378 INFO:teuthology.orchestra.run.trial028.stdout:Installing: 2026-02-13T02:18:17.378 INFO:teuthology.orchestra.run.trial028.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-13T02:18:17.379 INFO:teuthology.orchestra.run.trial028.stdout: nvmetcli noarch 0.8-3.el9 baseos 44 k 2026-02-13T02:18:17.379 INFO:teuthology.orchestra.run.trial028.stdout:Upgrading: 2026-02-13T02:18:17.379 INFO:teuthology.orchestra.run.trial028.stdout: libnvme x86_64 1.16.1-2.el9 baseos 116 k 2026-02-13T02:18:17.379 INFO:teuthology.orchestra.run.trial028.stdout:Installing dependencies: 2026-02-13T02:18:17.379 INFO:teuthology.orchestra.run.trial028.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2026-02-13T02:18:17.379 INFO:teuthology.orchestra.run.trial028.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2026-02-13T02:18:17.379 INFO:teuthology.orchestra.run.trial028.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-13T02:18:17.379 INFO:teuthology.orchestra.run.trial028.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2026-02-13T02:18:17.379 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:18:17.379 INFO:teuthology.orchestra.run.trial028.stdout:Transaction Summary 2026-02-13T02:18:17.379 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-13T02:18:17.379 INFO:teuthology.orchestra.run.trial028.stdout:Install 6 Packages 2026-02-13T02:18:17.380 INFO:teuthology.orchestra.run.trial028.stdout:Upgrade 1 Package 2026-02-13T02:18:17.380 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:18:17.380 INFO:teuthology.orchestra.run.trial028.stdout:Total download size: 2.4 M 2026-02-13T02:18:17.380 INFO:teuthology.orchestra.run.trial028.stdout:Downloading Packages: 2026-02-13T02:18:17.381 INFO:teuthology.orchestra.run.trial107.stdout:Dependencies resolved. 2026-02-13T02:18:17.381 INFO:teuthology.orchestra.run.trial107.stdout:================================================================================ 2026-02-13T02:18:17.382 INFO:teuthology.orchestra.run.trial107.stdout: Package Architecture Version Repository Size 2026-02-13T02:18:17.382 INFO:teuthology.orchestra.run.trial107.stdout:================================================================================ 2026-02-13T02:18:17.382 INFO:teuthology.orchestra.run.trial107.stdout:Installing: 2026-02-13T02:18:17.382 INFO:teuthology.orchestra.run.trial107.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-13T02:18:17.382 INFO:teuthology.orchestra.run.trial107.stdout: nvmetcli noarch 0.8-3.el9 baseos 44 k 2026-02-13T02:18:17.382 INFO:teuthology.orchestra.run.trial107.stdout:Upgrading: 2026-02-13T02:18:17.382 INFO:teuthology.orchestra.run.trial107.stdout: libnvme x86_64 1.16.1-2.el9 baseos 116 k 2026-02-13T02:18:17.382 INFO:teuthology.orchestra.run.trial107.stdout:Installing dependencies: 2026-02-13T02:18:17.382 INFO:teuthology.orchestra.run.trial107.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2026-02-13T02:18:17.382 INFO:teuthology.orchestra.run.trial107.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2026-02-13T02:18:17.382 INFO:teuthology.orchestra.run.trial107.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-13T02:18:17.383 INFO:teuthology.orchestra.run.trial107.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2026-02-13T02:18:17.383 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:18:17.383 INFO:teuthology.orchestra.run.trial107.stdout:Transaction Summary 2026-02-13T02:18:17.383 INFO:teuthology.orchestra.run.trial107.stdout:================================================================================ 2026-02-13T02:18:17.383 INFO:teuthology.orchestra.run.trial107.stdout:Install 6 Packages 2026-02-13T02:18:17.383 INFO:teuthology.orchestra.run.trial107.stdout:Upgrade 1 Package 2026-02-13T02:18:17.383 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:18:17.383 INFO:teuthology.orchestra.run.trial107.stdout:Total download size: 2.4 M 2026-02-13T02:18:17.383 INFO:teuthology.orchestra.run.trial107.stdout:Downloading Packages: 2026-02-13T02:18:18.042 INFO:teuthology.orchestra.run.trial028.stdout:(1/7): nvmetcli-0.8-3.el9.noarch.rpm 80 kB/s | 44 kB 00:00 2026-02-13T02:18:18.052 INFO:teuthology.orchestra.run.trial028.stdout:(2/7): python3-configshell-1.1.30-1.el9.noarch. 129 kB/s | 72 kB 00:00 2026-02-13T02:18:18.089 INFO:teuthology.orchestra.run.trial107.stdout:(1/7): nvmetcli-0.8-3.el9.noarch.rpm 73 kB/s | 44 kB 00:00 2026-02-13T02:18:18.093 INFO:teuthology.orchestra.run.trial107.stdout:(2/7): python3-configshell-1.1.30-1.el9.noarch. 119 kB/s | 72 kB 00:00 2026-02-13T02:18:18.164 INFO:teuthology.orchestra.run.trial027.stdout:(1/7): python3-configshell-1.1.30-1.el9.noarch. 106 kB/s | 72 kB 00:00 2026-02-13T02:18:18.223 INFO:teuthology.orchestra.run.trial028.stdout:(3/7): python3-kmod-0.9-32.el9.x86_64.rpm 467 kB/s | 84 kB 00:00 2026-02-13T02:18:18.234 INFO:teuthology.orchestra.run.trial027.stdout:(2/7): nvmetcli-0.8-3.el9.noarch.rpm 59 kB/s | 44 kB 00:00 2026-02-13T02:18:18.268 INFO:teuthology.orchestra.run.trial107.stdout:(3/7): python3-kmod-0.9-32.el9.x86_64.rpm 472 kB/s | 84 kB 00:00 2026-02-13T02:18:18.351 INFO:teuthology.orchestra.run.trial107.stdout:(4/7): python3-pyparsing-2.4.7-9.el9.noarch.rpm 583 kB/s | 150 kB 00:00 2026-02-13T02:18:18.383 INFO:teuthology.orchestra.run.trial028.stdout:(4/7): python3-pyparsing-2.4.7-9.el9.noarch.rpm 454 kB/s | 150 kB 00:00 2026-02-13T02:18:18.400 INFO:teuthology.orchestra.run.trial027.stdout:(3/7): python3-kmod-0.9-32.el9.x86_64.rpm 357 kB/s | 84 kB 00:00 2026-02-13T02:18:18.486 INFO:teuthology.orchestra.run.trial027.stdout:(4/7): python3-pyparsing-2.4.7-9.el9.noarch.rpm 597 kB/s | 150 kB 00:00 2026-02-13T02:18:18.532 INFO:teuthology.orchestra.run.trial107.stdout:(5/7): libnvme-1.16.1-2.el9.x86_64.rpm 643 kB/s | 116 kB 00:00 2026-02-13T02:18:18.560 INFO:teuthology.orchestra.run.trial028.stdout:(5/7): libnvme-1.16.1-2.el9.x86_64.rpm 659 kB/s | 116 kB 00:00 2026-02-13T02:18:18.734 INFO:teuthology.orchestra.run.trial027.stdout:(5/7): libnvme-1.16.1-2.el9.x86_64.rpm 469 kB/s | 116 kB 00:00 2026-02-13T02:18:19.274 INFO:teuthology.orchestra.run.trial028.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 797 kB/s | 837 kB 00:01 2026-02-13T02:18:19.337 INFO:teuthology.orchestra.run.trial107.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 783 kB/s | 837 kB 00:01 2026-02-13T02:18:19.373 INFO:teuthology.orchestra.run.trial028.stdout:(7/7): nvme-cli-2.16-1.el9.x86_64.rpm 627 kB/s | 1.2 MB 00:01 2026-02-13T02:18:19.375 INFO:teuthology.orchestra.run.trial028.stdout:-------------------------------------------------------------------------------- 2026-02-13T02:18:19.375 INFO:teuthology.orchestra.run.trial028.stdout:Total 1.2 MB/s | 2.4 MB 00:01 2026-02-13T02:18:19.408 INFO:teuthology.orchestra.run.trial027.stdout:(6/7): nvme-cli-2.16-1.el9.x86_64.rpm 614 kB/s | 1.2 MB 00:01 2026-02-13T02:18:19.440 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction check 2026-02-13T02:18:19.448 INFO:teuthology.orchestra.run.trial028.stdout:Transaction check succeeded. 2026-02-13T02:18:19.448 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction test 2026-02-13T02:18:19.493 INFO:teuthology.orchestra.run.trial027.stdout:(7/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 766 kB/s | 837 kB 00:01 2026-02-13T02:18:19.495 INFO:teuthology.orchestra.run.trial027.stdout:-------------------------------------------------------------------------------- 2026-02-13T02:18:19.495 INFO:teuthology.orchestra.run.trial027.stdout:Total 1.1 MB/s | 2.4 MB 00:02 2026-02-13T02:18:19.511 INFO:teuthology.orchestra.run.trial028.stdout:Transaction test succeeded. 2026-02-13T02:18:19.511 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction 2026-02-13T02:18:19.561 INFO:teuthology.orchestra.run.trial027.stdout:Running transaction check 2026-02-13T02:18:19.569 INFO:teuthology.orchestra.run.trial027.stdout:Transaction check succeeded. 2026-02-13T02:18:19.570 INFO:teuthology.orchestra.run.trial027.stdout:Running transaction test 2026-02-13T02:18:19.581 INFO:teuthology.orchestra.run.trial028.stdout: Preparing : 1/1 2026-02-13T02:18:19.632 INFO:teuthology.orchestra.run.trial027.stdout:Transaction test succeeded. 2026-02-13T02:18:19.632 INFO:teuthology.orchestra.run.trial027.stdout:Running transaction 2026-02-13T02:18:19.648 INFO:teuthology.orchestra.run.trial028.stdout: Upgrading : libnvme-1.16.1-2.el9.x86_64 1/8 2026-02-13T02:18:19.655 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 2/8 2026-02-13T02:18:19.664 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 3/8 2026-02-13T02:18:19.669 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 4/8 2026-02-13T02:18:19.675 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 5/8 2026-02-13T02:18:19.676 INFO:teuthology.orchestra.run.trial028.stdout: Installing : nvmetcli-0.8-3.el9.noarch 6/8 2026-02-13T02:18:19.702 INFO:teuthology.orchestra.run.trial027.stdout: Preparing : 1/1 2026-02-13T02:18:19.722 INFO:teuthology.orchestra.run.trial107.stdout:(7/7): nvme-cli-2.16-1.el9.x86_64.rpm 528 kB/s | 1.2 MB 00:02 2026-02-13T02:18:19.724 INFO:teuthology.orchestra.run.trial107.stdout:-------------------------------------------------------------------------------- 2026-02-13T02:18:19.725 INFO:teuthology.orchestra.run.trial107.stdout:Total 1.0 MB/s | 2.4 MB 00:02 2026-02-13T02:18:19.769 INFO:teuthology.orchestra.run.trial027.stdout: Upgrading : libnvme-1.16.1-2.el9.x86_64 1/8 2026-02-13T02:18:19.777 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 2/8 2026-02-13T02:18:19.785 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 3/8 2026-02-13T02:18:19.788 INFO:teuthology.orchestra.run.trial107.stdout:Running transaction check 2026-02-13T02:18:19.791 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 4/8 2026-02-13T02:18:19.796 INFO:teuthology.orchestra.run.trial107.stdout:Transaction check succeeded. 2026-02-13T02:18:19.796 INFO:teuthology.orchestra.run.trial107.stdout:Running transaction test 2026-02-13T02:18:19.797 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 5/8 2026-02-13T02:18:19.797 INFO:teuthology.orchestra.run.trial027.stdout: Installing : nvmetcli-0.8-3.el9.noarch 6/8 2026-02-13T02:18:19.807 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: nvmetcli-0.8-3.el9.noarch 6/8 2026-02-13T02:18:19.810 INFO:teuthology.orchestra.run.trial028.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 7/8 2026-02-13T02:18:19.858 INFO:teuthology.orchestra.run.trial107.stdout:Transaction test succeeded. 2026-02-13T02:18:19.859 INFO:teuthology.orchestra.run.trial107.stdout:Running transaction 2026-02-13T02:18:19.929 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: nvmetcli-0.8-3.el9.noarch 6/8 2026-02-13T02:18:19.930 INFO:teuthology.orchestra.run.trial107.stdout: Preparing : 1/1 2026-02-13T02:18:19.932 INFO:teuthology.orchestra.run.trial027.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 7/8 2026-02-13T02:18:19.997 INFO:teuthology.orchestra.run.trial107.stdout: Upgrading : libnvme-1.16.1-2.el9.x86_64 1/8 2026-02-13T02:18:20.004 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 2/8 2026-02-13T02:18:20.013 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 3/8 2026-02-13T02:18:20.018 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 4/8 2026-02-13T02:18:20.024 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 5/8 2026-02-13T02:18:20.025 INFO:teuthology.orchestra.run.trial107.stdout: Installing : nvmetcli-0.8-3.el9.noarch 6/8 2026-02-13T02:18:20.092 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 7/8 2026-02-13T02:18:20.092 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-13T02:18:20.092 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:18:20.116 INFO:teuthology.orchestra.run.trial028.stdout: Cleanup : libnvme-1.13-1.el9.x86_64 8/8 2026-02-13T02:18:20.156 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: nvmetcli-0.8-3.el9.noarch 6/8 2026-02-13T02:18:20.158 INFO:teuthology.orchestra.run.trial107.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 7/8 2026-02-13T02:18:20.216 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 7/8 2026-02-13T02:18:20.216 INFO:teuthology.orchestra.run.trial027.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-13T02:18:20.217 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:18:20.241 INFO:teuthology.orchestra.run.trial027.stdout: Cleanup : libnvme-1.13-1.el9.x86_64 8/8 2026-02-13T02:18:20.330 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: libnvme-1.13-1.el9.x86_64 8/8 2026-02-13T02:18:20.330 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 1/8 2026-02-13T02:18:20.330 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : nvmetcli-0.8-3.el9.noarch 2/8 2026-02-13T02:18:20.331 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 3/8 2026-02-13T02:18:20.331 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 4/8 2026-02-13T02:18:20.331 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 5/8 2026-02-13T02:18:20.331 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/8 2026-02-13T02:18:20.331 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libnvme-1.16.1-2.el9.x86_64 7/8 2026-02-13T02:18:20.382 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libnvme-1.13-1.el9.x86_64 8/8 2026-02-13T02:18:20.382 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:18:20.382 INFO:teuthology.orchestra.run.trial028.stdout:Upgraded: 2026-02-13T02:18:20.383 INFO:teuthology.orchestra.run.trial028.stdout: libnvme-1.16.1-2.el9.x86_64 2026-02-13T02:18:20.383 INFO:teuthology.orchestra.run.trial028.stdout:Installed: 2026-02-13T02:18:20.383 INFO:teuthology.orchestra.run.trial028.stdout: nvme-cli-2.16-1.el9.x86_64 nvmetcli-0.8-3.el9.noarch 2026-02-13T02:18:20.383 INFO:teuthology.orchestra.run.trial028.stdout: python3-configshell-1:1.1.30-1.el9.noarch python3-kmod-0.9-32.el9.x86_64 2026-02-13T02:18:20.383 INFO:teuthology.orchestra.run.trial028.stdout: python3-pyparsing-2.4.7-9.el9.noarch python3-urwid-2.1.2-4.el9.x86_64 2026-02-13T02:18:20.383 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:18:20.383 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-13T02:18:20.413 DEBUG:teuthology.parallel:result is None 2026-02-13T02:18:20.437 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 7/8 2026-02-13T02:18:20.438 INFO:teuthology.orchestra.run.trial107.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-13T02:18:20.438 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:18:20.458 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: libnvme-1.13-1.el9.x86_64 8/8 2026-02-13T02:18:20.458 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 1/8 2026-02-13T02:18:20.459 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : nvmetcli-0.8-3.el9.noarch 2/8 2026-02-13T02:18:20.459 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 3/8 2026-02-13T02:18:20.459 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 4/8 2026-02-13T02:18:20.459 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 5/8 2026-02-13T02:18:20.459 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/8 2026-02-13T02:18:20.459 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : libnvme-1.16.1-2.el9.x86_64 7/8 2026-02-13T02:18:20.461 INFO:teuthology.orchestra.run.trial107.stdout: Cleanup : libnvme-1.13-1.el9.x86_64 8/8 2026-02-13T02:18:20.512 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : libnvme-1.13-1.el9.x86_64 8/8 2026-02-13T02:18:20.512 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:18:20.512 INFO:teuthology.orchestra.run.trial027.stdout:Upgraded: 2026-02-13T02:18:20.512 INFO:teuthology.orchestra.run.trial027.stdout: libnvme-1.16.1-2.el9.x86_64 2026-02-13T02:18:20.512 INFO:teuthology.orchestra.run.trial027.stdout:Installed: 2026-02-13T02:18:20.512 INFO:teuthology.orchestra.run.trial027.stdout: nvme-cli-2.16-1.el9.x86_64 nvmetcli-0.8-3.el9.noarch 2026-02-13T02:18:20.513 INFO:teuthology.orchestra.run.trial027.stdout: python3-configshell-1:1.1.30-1.el9.noarch python3-kmod-0.9-32.el9.x86_64 2026-02-13T02:18:20.513 INFO:teuthology.orchestra.run.trial027.stdout: python3-pyparsing-2.4.7-9.el9.noarch python3-urwid-2.1.2-4.el9.x86_64 2026-02-13T02:18:20.513 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:18:20.513 INFO:teuthology.orchestra.run.trial027.stdout:Complete! 2026-02-13T02:18:20.544 DEBUG:teuthology.parallel:result is None 2026-02-13T02:18:20.680 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: libnvme-1.13-1.el9.x86_64 8/8 2026-02-13T02:18:20.680 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 1/8 2026-02-13T02:18:20.680 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : nvmetcli-0.8-3.el9.noarch 2/8 2026-02-13T02:18:20.680 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 3/8 2026-02-13T02:18:20.680 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 4/8 2026-02-13T02:18:20.680 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 5/8 2026-02-13T02:18:20.680 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/8 2026-02-13T02:18:20.681 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : libnvme-1.16.1-2.el9.x86_64 7/8 2026-02-13T02:18:20.733 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : libnvme-1.13-1.el9.x86_64 8/8 2026-02-13T02:18:20.734 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:18:20.734 INFO:teuthology.orchestra.run.trial107.stdout:Upgraded: 2026-02-13T02:18:20.734 INFO:teuthology.orchestra.run.trial107.stdout: libnvme-1.16.1-2.el9.x86_64 2026-02-13T02:18:20.734 INFO:teuthology.orchestra.run.trial107.stdout:Installed: 2026-02-13T02:18:20.734 INFO:teuthology.orchestra.run.trial107.stdout: nvme-cli-2.16-1.el9.x86_64 nvmetcli-0.8-3.el9.noarch 2026-02-13T02:18:20.734 INFO:teuthology.orchestra.run.trial107.stdout: python3-configshell-1:1.1.30-1.el9.noarch python3-kmod-0.9-32.el9.x86_64 2026-02-13T02:18:20.734 INFO:teuthology.orchestra.run.trial107.stdout: python3-pyparsing-2.4.7-9.el9.noarch python3-urwid-2.1.2-4.el9.x86_64 2026-02-13T02:18:20.734 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:18:20.734 INFO:teuthology.orchestra.run.trial107.stdout:Complete! 2026-02-13T02:18:20.765 DEBUG:teuthology.parallel:result is None 2026-02-13T02:18:20.765 INFO:teuthology.run_tasks:Running task install... 2026-02-13T02:18:20.785 DEBUG:teuthology.task.install:project ceph 2026-02-13T02:18:20.785 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8f8d38294e4db8750b4b1649f62ce37caa63080f'}} 2026-02-13T02:18:20.786 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8f8d38294e4db8750b4b1649f62ce37caa63080f'} 2026-02-13T02:18:20.786 INFO:teuthology.task.install:Using flavor: default 2026-02-13T02:18:20.796 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-13T02:18:20.796 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2026-02-13T02:18:20.797 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f8d38294e4db8750b4b1649f62ce37caa63080f', 'tag': None, 'wait_for_package': False} 2026-02-13T02:18:20.798 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f8d38294e4db8750b4b1649f62ce37caa63080f 2026-02-13T02:18:20.801 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f8d38294e4db8750b4b1649f62ce37caa63080f', 'tag': None, 'wait_for_package': False} 2026-02-13T02:18:20.801 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f8d38294e4db8750b4b1649f62ce37caa63080f 2026-02-13T02:18:20.803 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f8d38294e4db8750b4b1649f62ce37caa63080f', 'tag': None, 'wait_for_package': False} 2026-02-13T02:18:20.803 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f8d38294e4db8750b4b1649f62ce37caa63080f 2026-02-13T02:18:21.043 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20260210.171210/8f8d38294e4db8750b4b1649f62ce37caa63080f/centos/9/flavors/default/ 2026-02-13T02:18:21.043 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5241.g8f8d3829 2026-02-13T02:18:21.050 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20260210.171210/8f8d38294e4db8750b4b1649f62ce37caa63080f/centos/9/flavors/default/ 2026-02-13T02:18:21.050 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5241.g8f8d3829 2026-02-13T02:18:21.053 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20260210.171210/8f8d38294e4db8750b4b1649f62ce37caa63080f/centos/9/flavors/default/ 2026-02-13T02:18:21.053 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5241.g8f8d3829 2026-02-13T02:18:21.318 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20260210.171210/8f8d38294e4db8750b4b1649f62ce37caa63080f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20260210.171210/8f8d38294e4db8750b4b1649f62ce37caa63080f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20260210.171210/8f8d38294e4db8750b4b1649f62ce37caa63080f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-13T02:18:21.319 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-13T02:18:21.319 DEBUG:teuthology.orchestra.run.trial028:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-13T02:18:21.324 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20260210.171210/8f8d38294e4db8750b4b1649f62ce37caa63080f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20260210.171210/8f8d38294e4db8750b4b1649f62ce37caa63080f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20260210.171210/8f8d38294e4db8750b4b1649f62ce37caa63080f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-13T02:18:21.324 DEBUG:teuthology.orchestra.run.trial027:> set -ex 2026-02-13T02:18:21.324 DEBUG:teuthology.orchestra.run.trial027:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-13T02:18:21.347 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2026-02-13T02:18:21.347 DEBUG:teuthology.orchestra.run.trial028:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f8d38294e4db8750b4b1649f62ce37caa63080f/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-13T02:18:21.350 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20260210.171210/8f8d38294e4db8750b4b1649f62ce37caa63080f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20260210.171210/8f8d38294e4db8750b4b1649f62ce37caa63080f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20260210.171210/8f8d38294e4db8750b4b1649f62ce37caa63080f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-13T02:18:21.350 DEBUG:teuthology.orchestra.run.trial107:> set -ex 2026-02-13T02:18:21.350 DEBUG:teuthology.orchestra.run.trial107:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-13T02:18:21.353 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2026-02-13T02:18:21.353 DEBUG:teuthology.orchestra.run.trial027:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f8d38294e4db8750b4b1649f62ce37caa63080f/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-13T02:18:21.380 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2026-02-13T02:18:21.380 DEBUG:teuthology.orchestra.run.trial107:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f8d38294e4db8750b4b1649f62ce37caa63080f/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-13T02:18:21.415 DEBUG:teuthology.orchestra.run.trial028:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-13T02:18:21.420 DEBUG:teuthology.orchestra.run.trial027:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-13T02:18:21.447 DEBUG:teuthology.orchestra.run.trial107:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-13T02:18:21.487 DEBUG:teuthology.orchestra.run.trial028:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-13T02:18:21.493 DEBUG:teuthology.orchestra.run.trial027:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-13T02:18:21.522 DEBUG:teuthology.orchestra.run.trial107:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-13T02:18:21.550 INFO:teuthology.orchestra.run.trial028.stdout:check_obsoletes = 1 2026-02-13T02:18:21.552 DEBUG:teuthology.orchestra.run.trial028:> sudo yum clean all 2026-02-13T02:18:21.555 INFO:teuthology.orchestra.run.trial027.stdout:check_obsoletes = 1 2026-02-13T02:18:21.557 DEBUG:teuthology.orchestra.run.trial027:> sudo yum clean all 2026-02-13T02:18:21.585 INFO:teuthology.orchestra.run.trial107.stdout:check_obsoletes = 1 2026-02-13T02:18:21.587 DEBUG:teuthology.orchestra.run.trial107:> sudo yum clean all 2026-02-13T02:18:21.701 INFO:teuthology.orchestra.run.trial028.stdout:41 files removed 2026-02-13T02:18:21.706 INFO:teuthology.orchestra.run.trial027.stdout:41 files removed 2026-02-13T02:18:21.715 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-osd ceph-osd-classic ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2026-02-13T02:18:21.720 DEBUG:teuthology.orchestra.run.trial027:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-osd ceph-osd-classic ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2026-02-13T02:18:21.737 INFO:teuthology.orchestra.run.trial107.stdout:41 files removed 2026-02-13T02:18:21.752 DEBUG:teuthology.orchestra.run.trial107:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-osd ceph-osd-classic ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2026-02-13T02:18:22.211 INFO:teuthology.orchestra.run.trial027.stdout:ceph packages for x86_64 263 kB/s | 91 kB 00:00 2026-02-13T02:18:22.235 INFO:teuthology.orchestra.run.trial107.stdout:ceph packages for x86_64 270 kB/s | 91 kB 00:00 2026-02-13T02:18:22.549 INFO:teuthology.orchestra.run.trial027.stdout:ceph noarch packages 75 kB/s | 23 kB 00:00 2026-02-13T02:18:22.565 INFO:teuthology.orchestra.run.trial107.stdout:ceph noarch packages 77 kB/s | 23 kB 00:00 2026-02-13T02:18:22.843 INFO:teuthology.orchestra.run.trial027.stdout:ceph source packages 7.6 kB/s | 2.1 kB 00:00 2026-02-13T02:18:22.862 INFO:teuthology.orchestra.run.trial107.stdout:ceph source packages 7.5 kB/s | 2.1 kB 00:00 2026-02-13T02:18:23.698 INFO:teuthology.orchestra.run.trial107.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.9 MB 00:00 2026-02-13T02:18:24.376 INFO:teuthology.orchestra.run.trial028.stdout:ceph packages for x86_64 36 kB/s | 91 kB 00:02 2026-02-13T02:18:24.671 INFO:teuthology.orchestra.run.trial028.stdout:ceph noarch packages 87 kB/s | 23 kB 00:00 2026-02-13T02:18:24.884 INFO:teuthology.orchestra.run.trial028.stdout:ceph source packages 11 kB/s | 2.1 kB 00:00 2026-02-13T02:18:26.280 INFO:teuthology.orchestra.run.trial028.stdout:CentOS Stream 9 - BaseOS 6.5 MB/s | 8.9 MB 00:01 2026-02-13T02:18:26.725 INFO:teuthology.orchestra.run.trial107.stdout:CentOS Stream 9 - AppStream 11 MB/s | 27 MB 00:02 2026-02-13T02:18:31.895 INFO:teuthology.orchestra.run.trial027.stdout:CentOS Stream 9 - BaseOS 1.0 MB/s | 8.9 MB 00:09 2026-02-13T02:18:32.790 INFO:teuthology.orchestra.run.trial027.stdout:CentOS Stream 9 - AppStream 76 MB/s | 27 MB 00:00 2026-02-13T02:18:35.463 INFO:teuthology.orchestra.run.trial027.stdout:CentOS Stream 9 - CRB 30 MB/s | 7.8 MB 00:00 2026-02-13T02:18:37.145 INFO:teuthology.orchestra.run.trial027.stdout:CentOS Stream 9 - Extras packages 21 kB/s | 20 kB 00:00 2026-02-13T02:18:37.402 INFO:teuthology.orchestra.run.trial027.stdout:Extra Packages for Enterprise Linux 111 MB/s | 20 MB 00:00 2026-02-13T02:18:38.020 INFO:teuthology.orchestra.run.trial107.stdout:CentOS Stream 9 - CRB 899 kB/s | 7.8 MB 00:08 2026-02-13T02:18:38.837 INFO:teuthology.orchestra.run.trial107.stdout:CentOS Stream 9 - Extras packages 189 kB/s | 20 kB 00:00 2026-02-13T02:18:39.546 INFO:teuthology.orchestra.run.trial107.stdout:Extra Packages for Enterprise Linux 32 MB/s | 20 MB 00:00 2026-02-13T02:18:40.655 INFO:teuthology.orchestra.run.trial027.stdout:lab-extras 1.3 MB/s | 50 kB 00:00 2026-02-13T02:18:41.940 INFO:teuthology.orchestra.run.trial027.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2026-02-13T02:18:41.940 INFO:teuthology.orchestra.run.trial027.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2026-02-13T02:18:41.946 INFO:teuthology.orchestra.run.trial027.stdout:Package NetworkManager-1:1.54.1-1.el9.x86_64 is already installed. 2026-02-13T02:18:41.946 INFO:teuthology.orchestra.run.trial027.stdout:Package iproute-6.17.0-1.el9.x86_64 is already installed. 2026-02-13T02:18:41.947 INFO:teuthology.orchestra.run.trial027.stdout:Package util-linux-2.37.4-21.el9.x86_64 is already installed. 2026-02-13T02:18:41.947 INFO:teuthology.orchestra.run.trial027.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2026-02-13T02:18:41.948 INFO:teuthology.orchestra.run.trial027.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2026-02-13T02:18:41.949 INFO:teuthology.orchestra.run.trial027.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2026-02-13T02:18:41.950 INFO:teuthology.orchestra.run.trial027.stdout:Package xfsprogs-6.4.0-7.el9.x86_64 is already installed. 2026-02-13T02:18:41.977 INFO:teuthology.orchestra.run.trial027.stdout:Dependencies resolved. 2026-02-13T02:18:41.982 INFO:teuthology.orchestra.run.trial027.stdout:======================================================================================= 2026-02-13T02:18:41.982 INFO:teuthology.orchestra.run.trial027.stdout: Package Arch Version Repository Size 2026-02-13T02:18:41.982 INFO:teuthology.orchestra.run.trial027.stdout:======================================================================================= 2026-02-13T02:18:41.982 INFO:teuthology.orchestra.run.trial027.stdout:Installing: 2026-02-13T02:18:41.982 INFO:teuthology.orchestra.run.trial027.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2026-02-13T02:18:41.982 INFO:teuthology.orchestra.run.trial027.stdout: ceph x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 6.5 k 2026-02-13T02:18:41.982 INFO:teuthology.orchestra.run.trial027.stdout: ceph-base x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 5.2 M 2026-02-13T02:18:41.982 INFO:teuthology.orchestra.run.trial027.stdout: ceph-fuse x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 987 k 2026-02-13T02:18:41.982 INFO:teuthology.orchestra.run.trial027.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 147 k 2026-02-13T02:18:41.983 INFO:teuthology.orchestra.run.trial027.stdout: ceph-mgr x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 940 k 2026-02-13T02:18:41.983 INFO:teuthology.orchestra.run.trial027.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5241.g8f8d3829.el9 ceph-noarch 183 k 2026-02-13T02:18:41.983 INFO:teuthology.orchestra.run.trial027.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5241.g8f8d3829.el9 ceph-noarch 15 M 2026-02-13T02:18:41.983 INFO:teuthology.orchestra.run.trial027.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5241.g8f8d3829.el9 ceph-noarch 7.4 M 2026-02-13T02:18:41.983 INFO:teuthology.orchestra.run.trial027.stdout: ceph-mgr-rook noarch 2:20.3.0-5241.g8f8d3829.el9 ceph-noarch 50 k 2026-02-13T02:18:41.983 INFO:teuthology.orchestra.run.trial027.stdout: ceph-osd x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 166 k 2026-02-13T02:18:41.983 INFO:teuthology.orchestra.run.trial027.stdout: ceph-osd-classic x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 16 M 2026-02-13T02:18:41.983 INFO:teuthology.orchestra.run.trial027.stdout: ceph-radosgw x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 13 M 2026-02-13T02:18:41.983 INFO:teuthology.orchestra.run.trial027.stdout: ceph-test x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 78 M 2026-02-13T02:18:41.983 INFO:teuthology.orchestra.run.trial027.stdout: ceph-volume noarch 2:20.3.0-5241.g8f8d3829.el9 ceph-noarch 299 k 2026-02-13T02:18:41.983 INFO:teuthology.orchestra.run.trial027.stdout: cephadm noarch 2:20.3.0-5241.g8f8d3829.el9 ceph-noarch 1.0 M 2026-02-13T02:18:41.983 INFO:teuthology.orchestra.run.trial027.stdout: cephfs-mirror x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 243 k 2026-02-13T02:18:41.983 INFO:teuthology.orchestra.run.trial027.stdout: cephfs-shell x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 27 k 2026-02-13T02:18:41.983 INFO:teuthology.orchestra.run.trial027.stdout: cephfs-top noarch 2:20.3.0-5241.g8f8d3829.el9 ceph-noarch 18 k 2026-02-13T02:18:41.983 INFO:teuthology.orchestra.run.trial027.stdout: elfutils-libelf-devel x86_64 0.194-1.el9 appstream 79 k 2026-02-13T02:18:41.983 INFO:teuthology.orchestra.run.trial027.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2026-02-13T02:18:41.984 INFO:teuthology.orchestra.run.trial027.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2026-02-13T02:18:41.984 INFO:teuthology.orchestra.run.trial027.stdout: libcephfs-devel x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 37 k 2026-02-13T02:18:41.984 INFO:teuthology.orchestra.run.trial027.stdout: libcephfs2 x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 909 k 2026-02-13T02:18:41.984 INFO:teuthology.orchestra.run.trial027.stdout: librados-devel x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 123 k 2026-02-13T02:18:41.984 INFO:teuthology.orchestra.run.trial027.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2026-02-13T02:18:41.984 INFO:teuthology.orchestra.run.trial027.stdout: libuuid-devel x86_64 2.37.4-21.el9 appstream 25 k 2026-02-13T02:18:41.984 INFO:teuthology.orchestra.run.trial027.stdout: openssl-devel x86_64 1:3.5.5-1.el9 appstream 4.8 M 2026-02-13T02:18:41.984 INFO:teuthology.orchestra.run.trial027.stdout: postgresql x86_64 13.23-1.el9 appstream 1.6 M 2026-02-13T02:18:41.984 INFO:teuthology.orchestra.run.trial027.stdout: postgresql-contrib x86_64 13.23-1.el9 appstream 903 k 2026-02-13T02:18:41.984 INFO:teuthology.orchestra.run.trial027.stdout: postgresql-server x86_64 13.23-1.el9 appstream 5.8 M 2026-02-13T02:18:41.984 INFO:teuthology.orchestra.run.trial027.stdout: python3-cephfs x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 204 k 2026-02-13T02:18:41.984 INFO:teuthology.orchestra.run.trial027.stdout: python3-rados x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 324 k 2026-02-13T02:18:41.984 INFO:teuthology.orchestra.run.trial027.stdout: python3-rbd x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 306 k 2026-02-13T02:18:41.984 INFO:teuthology.orchestra.run.trial027.stdout: python3-rgw x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 100 k 2026-02-13T02:18:41.985 INFO:teuthology.orchestra.run.trial027.stdout: rbd-fuse x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 84 k 2026-02-13T02:18:41.985 INFO:teuthology.orchestra.run.trial027.stdout: rbd-mirror x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 2.9 M 2026-02-13T02:18:41.985 INFO:teuthology.orchestra.run.trial027.stdout: rbd-nbd x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 172 k 2026-02-13T02:18:41.985 INFO:teuthology.orchestra.run.trial027.stdout: xfsprogs-devel x86_64 6.4.0-7.el9 appstream 96 k 2026-02-13T02:18:41.985 INFO:teuthology.orchestra.run.trial027.stdout:Upgrading: 2026-02-13T02:18:41.985 INFO:teuthology.orchestra.run.trial027.stdout: NetworkManager x86_64 1:1.54.3-2.el9 baseos 2.4 M 2026-02-13T02:18:41.985 INFO:teuthology.orchestra.run.trial027.stdout: NetworkManager-libnm x86_64 1:1.54.3-2.el9 baseos 1.9 M 2026-02-13T02:18:41.985 INFO:teuthology.orchestra.run.trial027.stdout: NetworkManager-team x86_64 1:1.54.3-2.el9 baseos 37 k 2026-02-13T02:18:41.985 INFO:teuthology.orchestra.run.trial027.stdout: NetworkManager-tui x86_64 1:1.54.3-2.el9 baseos 248 k 2026-02-13T02:18:41.985 INFO:teuthology.orchestra.run.trial027.stdout: iproute x86_64 6.17.0-2.el9 baseos 866 k 2026-02-13T02:18:41.985 INFO:teuthology.orchestra.run.trial027.stdout: iproute-tc x86_64 6.17.0-2.el9 baseos 454 k 2026-02-13T02:18:41.985 INFO:teuthology.orchestra.run.trial027.stdout: librados2 x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 3.5 M 2026-02-13T02:18:41.985 INFO:teuthology.orchestra.run.trial027.stdout: librbd1 x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 2.8 M 2026-02-13T02:18:41.985 INFO:teuthology.orchestra.run.trial027.stdout: openssl x86_64 1:3.5.5-1.el9 baseos 1.5 M 2026-02-13T02:18:41.985 INFO:teuthology.orchestra.run.trial027.stdout: openssl-fips-provider x86_64 1:3.5.5-1.el9 baseos 814 k 2026-02-13T02:18:41.986 INFO:teuthology.orchestra.run.trial027.stdout: openssl-libs x86_64 1:3.5.5-1.el9 baseos 2.3 M 2026-02-13T02:18:41.986 INFO:teuthology.orchestra.run.trial027.stdout:Installing dependencies: 2026-02-13T02:18:41.986 INFO:teuthology.orchestra.run.trial027.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-13T02:18:41.986 INFO:teuthology.orchestra.run.trial027.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2026-02-13T02:18:41.986 INFO:teuthology.orchestra.run.trial027.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-13T02:18:41.986 INFO:teuthology.orchestra.run.trial027.stdout: ceph-common x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 24 M 2026-02-13T02:18:41.986 INFO:teuthology.orchestra.run.trial027.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5241.g8f8d3829.el9 ceph-noarch 57 k 2026-02-13T02:18:41.986 INFO:teuthology.orchestra.run.trial027.stdout: ceph-mds x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 2.4 M 2026-02-13T02:18:41.986 INFO:teuthology.orchestra.run.trial027.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5241.g8f8d3829.el9 ceph-noarch 297 k 2026-02-13T02:18:41.986 INFO:teuthology.orchestra.run.trial027.stdout: ceph-mon x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 5.0 M 2026-02-13T02:18:41.986 INFO:teuthology.orchestra.run.trial027.stdout: ceph-osd-crimson x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 19 M 2026-02-13T02:18:41.986 INFO:teuthology.orchestra.run.trial027.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5241.g8f8d3829.el9 ceph-noarch 17 k 2026-02-13T02:18:41.986 INFO:teuthology.orchestra.run.trial027.stdout: ceph-selinux x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 25 k 2026-02-13T02:18:41.986 INFO:teuthology.orchestra.run.trial027.stdout: cryptsetup x86_64 2.8.1-2.el9 baseos 350 k 2026-02-13T02:18:41.986 INFO:teuthology.orchestra.run.trial027.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2026-02-13T02:18:41.986 INFO:teuthology.orchestra.run.trial027.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2026-02-13T02:18:41.986 INFO:teuthology.orchestra.run.trial027.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2026-02-13T02:18:41.987 INFO:teuthology.orchestra.run.trial027.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-13T02:18:41.987 INFO:teuthology.orchestra.run.trial027.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-13T02:18:41.987 INFO:teuthology.orchestra.run.trial027.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-13T02:18:41.987 INFO:teuthology.orchestra.run.trial027.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-13T02:18:41.987 INFO:teuthology.orchestra.run.trial027.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-13T02:18:41.987 INFO:teuthology.orchestra.run.trial027.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-13T02:18:41.987 INFO:teuthology.orchestra.run.trial027.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-13T02:18:41.987 INFO:teuthology.orchestra.run.trial027.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 24 k 2026-02-13T02:18:41.987 INFO:teuthology.orchestra.run.trial027.stdout: libcephsqlite x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 159 k 2026-02-13T02:18:41.987 INFO:teuthology.orchestra.run.trial027.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-13T02:18:41.987 INFO:teuthology.orchestra.run.trial027.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-13T02:18:41.987 INFO:teuthology.orchestra.run.trial027.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-13T02:18:41.987 INFO:teuthology.orchestra.run.trial027.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-13T02:18:41.987 INFO:teuthology.orchestra.run.trial027.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-13T02:18:41.987 INFO:teuthology.orchestra.run.trial027.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-13T02:18:41.987 INFO:teuthology.orchestra.run.trial027.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-13T02:18:41.987 INFO:teuthology.orchestra.run.trial027.stdout: libradosstriper1 x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 248 k 2026-02-13T02:18:41.987 INFO:teuthology.orchestra.run.trial027.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-13T02:18:41.988 INFO:teuthology.orchestra.run.trial027.stdout: librgw2 x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 6.5 M 2026-02-13T02:18:41.988 INFO:teuthology.orchestra.run.trial027.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-13T02:18:41.988 INFO:teuthology.orchestra.run.trial027.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-13T02:18:41.988 INFO:teuthology.orchestra.run.trial027.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-13T02:18:41.988 INFO:teuthology.orchestra.run.trial027.stdout: libzstd-devel x86_64 1.5.5-1.el9 appstream 51 k 2026-02-13T02:18:41.988 INFO:teuthology.orchestra.run.trial027.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-13T02:18:41.988 INFO:teuthology.orchestra.run.trial027.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-13T02:18:41.988 INFO:teuthology.orchestra.run.trial027.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-13T02:18:41.988 INFO:teuthology.orchestra.run.trial027.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-13T02:18:41.988 INFO:teuthology.orchestra.run.trial027.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-13T02:18:41.988 INFO:teuthology.orchestra.run.trial027.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-13T02:18:41.988 INFO:teuthology.orchestra.run.trial027.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-13T02:18:41.988 INFO:teuthology.orchestra.run.trial027.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-13T02:18:41.988 INFO:teuthology.orchestra.run.trial027.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-13T02:18:41.988 INFO:teuthology.orchestra.run.trial027.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-13T02:18:41.988 INFO:teuthology.orchestra.run.trial027.stdout: perl-Thread-Queue noarch 3.14-460.el9 appstream 22 k 2026-02-13T02:18:41.989 INFO:teuthology.orchestra.run.trial027.stdout: perl-threads x86_64 1:2.25-460.el9 appstream 58 k 2026-02-13T02:18:41.989 INFO:teuthology.orchestra.run.trial027.stdout: perl-threads-shared x86_64 1.61-460.el9 appstream 45 k 2026-02-13T02:18:41.989 INFO:teuthology.orchestra.run.trial027.stdout: postgresql-private-libs x86_64 13.23-1.el9 appstream 138 k 2026-02-13T02:18:41.989 INFO:teuthology.orchestra.run.trial027.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-13T02:18:41.989 INFO:teuthology.orchestra.run.trial027.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-13T02:18:41.989 INFO:teuthology.orchestra.run.trial027.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-13T02:18:41.989 INFO:teuthology.orchestra.run.trial027.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-13T02:18:41.989 INFO:teuthology.orchestra.run.trial027.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-13T02:18:41.989 INFO:teuthology.orchestra.run.trial027.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-13T02:18:41.989 INFO:teuthology.orchestra.run.trial027.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-13T02:18:41.989 INFO:teuthology.orchestra.run.trial027.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-13T02:18:41.989 INFO:teuthology.orchestra.run.trial027.stdout: python3-ceph-argparse x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 45 k 2026-02-13T02:18:41.989 INFO:teuthology.orchestra.run.trial027.stdout: python3-ceph-common x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 185 k 2026-02-13T02:18:41.989 INFO:teuthology.orchestra.run.trial027.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-13T02:18:41.989 INFO:teuthology.orchestra.run.trial027.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-13T02:18:41.989 INFO:teuthology.orchestra.run.trial027.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-13T02:18:41.989 INFO:teuthology.orchestra.run.trial027.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-13T02:18:41.989 INFO:teuthology.orchestra.run.trial027.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2026-02-13T02:18:41.989 INFO:teuthology.orchestra.run.trial027.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2026-02-13T02:18:41.990 INFO:teuthology.orchestra.run.trial027.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-13T02:18:41.990 INFO:teuthology.orchestra.run.trial027.stdout: python3-devel x86_64 3.9.25-2.el9 appstream 244 k 2026-02-13T02:18:41.990 INFO:teuthology.orchestra.run.trial027.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-13T02:18:41.990 INFO:teuthology.orchestra.run.trial027.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-13T02:18:41.990 INFO:teuthology.orchestra.run.trial027.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-13T02:18:41.990 INFO:teuthology.orchestra.run.trial027.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-13T02:18:41.990 INFO:teuthology.orchestra.run.trial027.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-13T02:18:41.990 INFO:teuthology.orchestra.run.trial027.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-13T02:18:41.990 INFO:teuthology.orchestra.run.trial027.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-13T02:18:41.990 INFO:teuthology.orchestra.run.trial027.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-13T02:18:41.990 INFO:teuthology.orchestra.run.trial027.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-13T02:18:41.990 INFO:teuthology.orchestra.run.trial027.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-13T02:18:41.990 INFO:teuthology.orchestra.run.trial027.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-13T02:18:41.990 INFO:teuthology.orchestra.run.trial027.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-13T02:18:41.990 INFO:teuthology.orchestra.run.trial027.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-13T02:18:41.990 INFO:teuthology.orchestra.run.trial027.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-13T02:18:41.990 INFO:teuthology.orchestra.run.trial027.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-13T02:18:41.990 INFO:teuthology.orchestra.run.trial027.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-13T02:18:41.990 INFO:teuthology.orchestra.run.trial027.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-13T02:18:41.990 INFO:teuthology.orchestra.run.trial027.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-13T02:18:41.991 INFO:teuthology.orchestra.run.trial027.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-13T02:18:41.991 INFO:teuthology.orchestra.run.trial027.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-13T02:18:41.991 INFO:teuthology.orchestra.run.trial027.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-13T02:18:41.991 INFO:teuthology.orchestra.run.trial027.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-13T02:18:41.991 INFO:teuthology.orchestra.run.trial027.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-13T02:18:41.991 INFO:teuthology.orchestra.run.trial027.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-13T02:18:41.991 INFO:teuthology.orchestra.run.trial027.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-13T02:18:41.991 INFO:teuthology.orchestra.run.trial027.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-13T02:18:41.991 INFO:teuthology.orchestra.run.trial027.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2026-02-13T02:18:41.991 INFO:teuthology.orchestra.run.trial027.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-13T02:18:41.991 INFO:teuthology.orchestra.run.trial027.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-13T02:18:41.991 INFO:teuthology.orchestra.run.trial027.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-13T02:18:41.991 INFO:teuthology.orchestra.run.trial027.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-13T02:18:41.991 INFO:teuthology.orchestra.run.trial027.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-13T02:18:41.991 INFO:teuthology.orchestra.run.trial027.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-13T02:18:41.991 INFO:teuthology.orchestra.run.trial027.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-13T02:18:41.991 INFO:teuthology.orchestra.run.trial027.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-13T02:18:41.991 INFO:teuthology.orchestra.run.trial027.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-13T02:18:41.991 INFO:teuthology.orchestra.run.trial027.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-13T02:18:41.991 INFO:teuthology.orchestra.run.trial027.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2026-02-13T02:18:41.992 INFO:teuthology.orchestra.run.trial027.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-13T02:18:41.992 INFO:teuthology.orchestra.run.trial027.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-13T02:18:41.992 INFO:teuthology.orchestra.run.trial027.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-13T02:18:41.992 INFO:teuthology.orchestra.run.trial027.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-13T02:18:41.992 INFO:teuthology.orchestra.run.trial027.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-13T02:18:41.992 INFO:teuthology.orchestra.run.trial027.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-13T02:18:41.992 INFO:teuthology.orchestra.run.trial027.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-13T02:18:41.992 INFO:teuthology.orchestra.run.trial027.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-13T02:18:41.992 INFO:teuthology.orchestra.run.trial027.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-13T02:18:41.992 INFO:teuthology.orchestra.run.trial027.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2026-02-13T02:18:41.992 INFO:teuthology.orchestra.run.trial027.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-13T02:18:41.992 INFO:teuthology.orchestra.run.trial027.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-13T02:18:41.992 INFO:teuthology.orchestra.run.trial027.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-13T02:18:41.992 INFO:teuthology.orchestra.run.trial027.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2026-02-13T02:18:41.992 INFO:teuthology.orchestra.run.trial027.stdout:Installing weak dependencies: 2026-02-13T02:18:41.992 INFO:teuthology.orchestra.run.trial027.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-13T02:18:41.992 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:18:41.992 INFO:teuthology.orchestra.run.trial027.stdout:Transaction Summary 2026-02-13T02:18:41.992 INFO:teuthology.orchestra.run.trial027.stdout:======================================================================================= 2026-02-13T02:18:41.992 INFO:teuthology.orchestra.run.trial027.stdout:Install 162 Packages 2026-02-13T02:18:41.992 INFO:teuthology.orchestra.run.trial027.stdout:Upgrade 11 Packages 2026-02-13T02:18:41.993 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:18:41.993 INFO:teuthology.orchestra.run.trial027.stdout:Total download size: 302 M 2026-02-13T02:18:41.993 INFO:teuthology.orchestra.run.trial027.stdout:Downloading Packages: 2026-02-13T02:18:42.704 INFO:teuthology.orchestra.run.trial027.stdout:(1/173): ceph-20.3.0-5241.g8f8d3829.el9.x86_64. 61 kB/s | 6.5 kB 00:00 2026-02-13T02:18:42.740 INFO:teuthology.orchestra.run.trial107.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-13T02:18:42.868 INFO:teuthology.orchestra.run.trial027.stdout:(2/173): ceph-fuse-20.3.0-5241.g8f8d3829.el9.x8 5.9 MB/s | 987 kB 00:00 2026-02-13T02:18:42.890 INFO:teuthology.orchestra.run.trial027.stdout:(3/173): ceph-immutable-object-cache-20.3.0-524 6.5 MB/s | 147 kB 00:00 2026-02-13T02:18:43.038 INFO:teuthology.orchestra.run.trial027.stdout:(4/173): ceph-mds-20.3.0-5241.g8f8d3829.el9.x86 16 MB/s | 2.4 MB 00:00 2026-02-13T02:18:43.048 INFO:teuthology.orchestra.run.trial027.stdout:(5/173): ceph-base-20.3.0-5241.g8f8d3829.el9.x8 12 MB/s | 5.2 MB 00:00 2026-02-13T02:18:43.106 INFO:teuthology.orchestra.run.trial027.stdout:(6/173): ceph-mgr-20.3.0-5241.g8f8d3829.el9.x86 13 MB/s | 940 kB 00:00 2026-02-13T02:18:43.128 INFO:teuthology.orchestra.run.trial027.stdout:(7/173): ceph-osd-20.3.0-5241.g8f8d3829.el9.x86 7.3 MB/s | 166 kB 00:00 2026-02-13T02:18:43.302 INFO:teuthology.orchestra.run.trial027.stdout:(8/173): ceph-mon-20.3.0-5241.g8f8d3829.el9.x86 20 MB/s | 5.0 MB 00:00 2026-02-13T02:18:43.940 INFO:teuthology.orchestra.run.trial027.stdout:(9/173): ceph-common-20.3.0-5241.g8f8d3829.el9. 18 MB/s | 24 MB 00:01 2026-02-13T02:18:44.019 INFO:teuthology.orchestra.run.trial107.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2026-02-13T02:18:44.019 INFO:teuthology.orchestra.run.trial107.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2026-02-13T02:18:44.025 INFO:teuthology.orchestra.run.trial107.stdout:Package NetworkManager-1:1.54.1-1.el9.x86_64 is already installed. 2026-02-13T02:18:44.025 INFO:teuthology.orchestra.run.trial107.stdout:Package iproute-6.17.0-1.el9.x86_64 is already installed. 2026-02-13T02:18:44.026 INFO:teuthology.orchestra.run.trial107.stdout:Package util-linux-2.37.4-21.el9.x86_64 is already installed. 2026-02-13T02:18:44.026 INFO:teuthology.orchestra.run.trial107.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2026-02-13T02:18:44.027 INFO:teuthology.orchestra.run.trial107.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2026-02-13T02:18:44.028 INFO:teuthology.orchestra.run.trial107.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2026-02-13T02:18:44.029 INFO:teuthology.orchestra.run.trial107.stdout:Package xfsprogs-6.4.0-7.el9.x86_64 is already installed. 2026-02-13T02:18:44.056 INFO:teuthology.orchestra.run.trial107.stdout:Dependencies resolved. 2026-02-13T02:18:44.061 INFO:teuthology.orchestra.run.trial107.stdout:======================================================================================= 2026-02-13T02:18:44.061 INFO:teuthology.orchestra.run.trial107.stdout: Package Arch Version Repository Size 2026-02-13T02:18:44.061 INFO:teuthology.orchestra.run.trial107.stdout:======================================================================================= 2026-02-13T02:18:44.061 INFO:teuthology.orchestra.run.trial107.stdout:Installing: 2026-02-13T02:18:44.061 INFO:teuthology.orchestra.run.trial107.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2026-02-13T02:18:44.061 INFO:teuthology.orchestra.run.trial107.stdout: ceph x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 6.5 k 2026-02-13T02:18:44.061 INFO:teuthology.orchestra.run.trial107.stdout: ceph-base x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 5.2 M 2026-02-13T02:18:44.061 INFO:teuthology.orchestra.run.trial107.stdout: ceph-fuse x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 987 k 2026-02-13T02:18:44.061 INFO:teuthology.orchestra.run.trial107.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 147 k 2026-02-13T02:18:44.061 INFO:teuthology.orchestra.run.trial107.stdout: ceph-mgr x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 940 k 2026-02-13T02:18:44.062 INFO:teuthology.orchestra.run.trial107.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5241.g8f8d3829.el9 ceph-noarch 183 k 2026-02-13T02:18:44.062 INFO:teuthology.orchestra.run.trial107.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5241.g8f8d3829.el9 ceph-noarch 15 M 2026-02-13T02:18:44.062 INFO:teuthology.orchestra.run.trial107.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5241.g8f8d3829.el9 ceph-noarch 7.4 M 2026-02-13T02:18:44.062 INFO:teuthology.orchestra.run.trial107.stdout: ceph-mgr-rook noarch 2:20.3.0-5241.g8f8d3829.el9 ceph-noarch 50 k 2026-02-13T02:18:44.062 INFO:teuthology.orchestra.run.trial107.stdout: ceph-osd x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 166 k 2026-02-13T02:18:44.062 INFO:teuthology.orchestra.run.trial107.stdout: ceph-osd-classic x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 16 M 2026-02-13T02:18:44.062 INFO:teuthology.orchestra.run.trial107.stdout: ceph-radosgw x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 13 M 2026-02-13T02:18:44.062 INFO:teuthology.orchestra.run.trial107.stdout: ceph-test x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 78 M 2026-02-13T02:18:44.062 INFO:teuthology.orchestra.run.trial107.stdout: ceph-volume noarch 2:20.3.0-5241.g8f8d3829.el9 ceph-noarch 299 k 2026-02-13T02:18:44.062 INFO:teuthology.orchestra.run.trial107.stdout: cephadm noarch 2:20.3.0-5241.g8f8d3829.el9 ceph-noarch 1.0 M 2026-02-13T02:18:44.062 INFO:teuthology.orchestra.run.trial107.stdout: cephfs-mirror x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 243 k 2026-02-13T02:18:44.062 INFO:teuthology.orchestra.run.trial107.stdout: cephfs-shell x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 27 k 2026-02-13T02:18:44.062 INFO:teuthology.orchestra.run.trial107.stdout: cephfs-top noarch 2:20.3.0-5241.g8f8d3829.el9 ceph-noarch 18 k 2026-02-13T02:18:44.063 INFO:teuthology.orchestra.run.trial107.stdout: elfutils-libelf-devel x86_64 0.194-1.el9 appstream 79 k 2026-02-13T02:18:44.063 INFO:teuthology.orchestra.run.trial107.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2026-02-13T02:18:44.063 INFO:teuthology.orchestra.run.trial107.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2026-02-13T02:18:44.063 INFO:teuthology.orchestra.run.trial107.stdout: libcephfs-devel x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 37 k 2026-02-13T02:18:44.063 INFO:teuthology.orchestra.run.trial107.stdout: libcephfs2 x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 909 k 2026-02-13T02:18:44.063 INFO:teuthology.orchestra.run.trial107.stdout: librados-devel x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 123 k 2026-02-13T02:18:44.063 INFO:teuthology.orchestra.run.trial107.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2026-02-13T02:18:44.063 INFO:teuthology.orchestra.run.trial107.stdout: libuuid-devel x86_64 2.37.4-21.el9 appstream 25 k 2026-02-13T02:18:44.063 INFO:teuthology.orchestra.run.trial107.stdout: openssl-devel x86_64 1:3.5.5-1.el9 appstream 4.8 M 2026-02-13T02:18:44.063 INFO:teuthology.orchestra.run.trial107.stdout: postgresql x86_64 13.23-1.el9 appstream 1.6 M 2026-02-13T02:18:44.063 INFO:teuthology.orchestra.run.trial107.stdout: postgresql-contrib x86_64 13.23-1.el9 appstream 903 k 2026-02-13T02:18:44.063 INFO:teuthology.orchestra.run.trial107.stdout: postgresql-server x86_64 13.23-1.el9 appstream 5.8 M 2026-02-13T02:18:44.064 INFO:teuthology.orchestra.run.trial107.stdout: python3-cephfs x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 204 k 2026-02-13T02:18:44.064 INFO:teuthology.orchestra.run.trial107.stdout: python3-rados x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 324 k 2026-02-13T02:18:44.064 INFO:teuthology.orchestra.run.trial107.stdout: python3-rbd x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 306 k 2026-02-13T02:18:44.064 INFO:teuthology.orchestra.run.trial107.stdout: python3-rgw x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 100 k 2026-02-13T02:18:44.064 INFO:teuthology.orchestra.run.trial107.stdout: rbd-fuse x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 84 k 2026-02-13T02:18:44.064 INFO:teuthology.orchestra.run.trial107.stdout: rbd-mirror x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 2.9 M 2026-02-13T02:18:44.064 INFO:teuthology.orchestra.run.trial107.stdout: rbd-nbd x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 172 k 2026-02-13T02:18:44.064 INFO:teuthology.orchestra.run.trial107.stdout: xfsprogs-devel x86_64 6.4.0-7.el9 appstream 96 k 2026-02-13T02:18:44.064 INFO:teuthology.orchestra.run.trial107.stdout:Upgrading: 2026-02-13T02:18:44.064 INFO:teuthology.orchestra.run.trial107.stdout: NetworkManager x86_64 1:1.54.3-2.el9 baseos 2.4 M 2026-02-13T02:18:44.064 INFO:teuthology.orchestra.run.trial107.stdout: NetworkManager-libnm x86_64 1:1.54.3-2.el9 baseos 1.9 M 2026-02-13T02:18:44.064 INFO:teuthology.orchestra.run.trial107.stdout: NetworkManager-team x86_64 1:1.54.3-2.el9 baseos 37 k 2026-02-13T02:18:44.065 INFO:teuthology.orchestra.run.trial107.stdout: NetworkManager-tui x86_64 1:1.54.3-2.el9 baseos 248 k 2026-02-13T02:18:44.065 INFO:teuthology.orchestra.run.trial107.stdout: iproute x86_64 6.17.0-2.el9 baseos 866 k 2026-02-13T02:18:44.065 INFO:teuthology.orchestra.run.trial107.stdout: iproute-tc x86_64 6.17.0-2.el9 baseos 454 k 2026-02-13T02:18:44.065 INFO:teuthology.orchestra.run.trial107.stdout: librados2 x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 3.5 M 2026-02-13T02:18:44.065 INFO:teuthology.orchestra.run.trial107.stdout: librbd1 x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 2.8 M 2026-02-13T02:18:44.065 INFO:teuthology.orchestra.run.trial107.stdout: openssl x86_64 1:3.5.5-1.el9 baseos 1.5 M 2026-02-13T02:18:44.065 INFO:teuthology.orchestra.run.trial107.stdout: openssl-fips-provider x86_64 1:3.5.5-1.el9 baseos 814 k 2026-02-13T02:18:44.065 INFO:teuthology.orchestra.run.trial107.stdout: openssl-libs x86_64 1:3.5.5-1.el9 baseos 2.3 M 2026-02-13T02:18:44.065 INFO:teuthology.orchestra.run.trial107.stdout:Installing dependencies: 2026-02-13T02:18:44.065 INFO:teuthology.orchestra.run.trial107.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-13T02:18:44.065 INFO:teuthology.orchestra.run.trial107.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2026-02-13T02:18:44.065 INFO:teuthology.orchestra.run.trial107.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-13T02:18:44.065 INFO:teuthology.orchestra.run.trial107.stdout: ceph-common x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 24 M 2026-02-13T02:18:44.066 INFO:teuthology.orchestra.run.trial107.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5241.g8f8d3829.el9 ceph-noarch 57 k 2026-02-13T02:18:44.066 INFO:teuthology.orchestra.run.trial107.stdout: ceph-mds x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 2.4 M 2026-02-13T02:18:44.066 INFO:teuthology.orchestra.run.trial107.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5241.g8f8d3829.el9 ceph-noarch 297 k 2026-02-13T02:18:44.066 INFO:teuthology.orchestra.run.trial107.stdout: ceph-mon x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 5.0 M 2026-02-13T02:18:44.066 INFO:teuthology.orchestra.run.trial107.stdout: ceph-osd-crimson x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 19 M 2026-02-13T02:18:44.066 INFO:teuthology.orchestra.run.trial107.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5241.g8f8d3829.el9 ceph-noarch 17 k 2026-02-13T02:18:44.066 INFO:teuthology.orchestra.run.trial107.stdout: ceph-selinux x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 25 k 2026-02-13T02:18:44.066 INFO:teuthology.orchestra.run.trial107.stdout: cryptsetup x86_64 2.8.1-2.el9 baseos 350 k 2026-02-13T02:18:44.066 INFO:teuthology.orchestra.run.trial107.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2026-02-13T02:18:44.066 INFO:teuthology.orchestra.run.trial107.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2026-02-13T02:18:44.066 INFO:teuthology.orchestra.run.trial107.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2026-02-13T02:18:44.066 INFO:teuthology.orchestra.run.trial107.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-13T02:18:44.066 INFO:teuthology.orchestra.run.trial107.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-13T02:18:44.067 INFO:teuthology.orchestra.run.trial107.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-13T02:18:44.067 INFO:teuthology.orchestra.run.trial107.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-13T02:18:44.067 INFO:teuthology.orchestra.run.trial107.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-13T02:18:44.067 INFO:teuthology.orchestra.run.trial107.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-13T02:18:44.067 INFO:teuthology.orchestra.run.trial107.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-13T02:18:44.067 INFO:teuthology.orchestra.run.trial107.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 24 k 2026-02-13T02:18:44.067 INFO:teuthology.orchestra.run.trial107.stdout: libcephsqlite x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 159 k 2026-02-13T02:18:44.067 INFO:teuthology.orchestra.run.trial107.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-13T02:18:44.067 INFO:teuthology.orchestra.run.trial107.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-13T02:18:44.067 INFO:teuthology.orchestra.run.trial107.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-13T02:18:44.067 INFO:teuthology.orchestra.run.trial107.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-13T02:18:44.067 INFO:teuthology.orchestra.run.trial107.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-13T02:18:44.067 INFO:teuthology.orchestra.run.trial107.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-13T02:18:44.068 INFO:teuthology.orchestra.run.trial107.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-13T02:18:44.068 INFO:teuthology.orchestra.run.trial107.stdout: libradosstriper1 x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 248 k 2026-02-13T02:18:44.068 INFO:teuthology.orchestra.run.trial107.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-13T02:18:44.068 INFO:teuthology.orchestra.run.trial107.stdout: librgw2 x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 6.5 M 2026-02-13T02:18:44.068 INFO:teuthology.orchestra.run.trial107.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-13T02:18:44.068 INFO:teuthology.orchestra.run.trial107.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-13T02:18:44.068 INFO:teuthology.orchestra.run.trial107.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-13T02:18:44.068 INFO:teuthology.orchestra.run.trial107.stdout: libzstd-devel x86_64 1.5.5-1.el9 appstream 51 k 2026-02-13T02:18:44.068 INFO:teuthology.orchestra.run.trial107.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-13T02:18:44.068 INFO:teuthology.orchestra.run.trial107.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-13T02:18:44.068 INFO:teuthology.orchestra.run.trial107.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-13T02:18:44.068 INFO:teuthology.orchestra.run.trial107.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-13T02:18:44.069 INFO:teuthology.orchestra.run.trial107.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-13T02:18:44.069 INFO:teuthology.orchestra.run.trial107.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-13T02:18:44.069 INFO:teuthology.orchestra.run.trial107.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-13T02:18:44.069 INFO:teuthology.orchestra.run.trial107.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-13T02:18:44.069 INFO:teuthology.orchestra.run.trial107.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-13T02:18:44.069 INFO:teuthology.orchestra.run.trial107.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-13T02:18:44.069 INFO:teuthology.orchestra.run.trial107.stdout: perl-Thread-Queue noarch 3.14-460.el9 appstream 22 k 2026-02-13T02:18:44.069 INFO:teuthology.orchestra.run.trial107.stdout: perl-threads x86_64 1:2.25-460.el9 appstream 58 k 2026-02-13T02:18:44.069 INFO:teuthology.orchestra.run.trial107.stdout: perl-threads-shared x86_64 1.61-460.el9 appstream 45 k 2026-02-13T02:18:44.069 INFO:teuthology.orchestra.run.trial107.stdout: postgresql-private-libs x86_64 13.23-1.el9 appstream 138 k 2026-02-13T02:18:44.069 INFO:teuthology.orchestra.run.trial107.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-13T02:18:44.069 INFO:teuthology.orchestra.run.trial107.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-13T02:18:44.069 INFO:teuthology.orchestra.run.trial107.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-13T02:18:44.070 INFO:teuthology.orchestra.run.trial107.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-13T02:18:44.070 INFO:teuthology.orchestra.run.trial107.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-13T02:18:44.070 INFO:teuthology.orchestra.run.trial107.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-13T02:18:44.070 INFO:teuthology.orchestra.run.trial107.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-13T02:18:44.070 INFO:teuthology.orchestra.run.trial107.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-13T02:18:44.070 INFO:teuthology.orchestra.run.trial107.stdout: python3-ceph-argparse x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 45 k 2026-02-13T02:18:44.070 INFO:teuthology.orchestra.run.trial107.stdout: python3-ceph-common x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 185 k 2026-02-13T02:18:44.070 INFO:teuthology.orchestra.run.trial107.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-13T02:18:44.070 INFO:teuthology.orchestra.run.trial107.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-13T02:18:44.070 INFO:teuthology.orchestra.run.trial107.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-13T02:18:44.070 INFO:teuthology.orchestra.run.trial107.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-13T02:18:44.070 INFO:teuthology.orchestra.run.trial107.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2026-02-13T02:18:44.070 INFO:teuthology.orchestra.run.trial107.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2026-02-13T02:18:44.071 INFO:teuthology.orchestra.run.trial107.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-13T02:18:44.071 INFO:teuthology.orchestra.run.trial107.stdout: python3-devel x86_64 3.9.25-2.el9 appstream 244 k 2026-02-13T02:18:44.071 INFO:teuthology.orchestra.run.trial107.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-13T02:18:44.071 INFO:teuthology.orchestra.run.trial107.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-13T02:18:44.071 INFO:teuthology.orchestra.run.trial107.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-13T02:18:44.071 INFO:teuthology.orchestra.run.trial107.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-13T02:18:44.071 INFO:teuthology.orchestra.run.trial107.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-13T02:18:44.071 INFO:teuthology.orchestra.run.trial107.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-13T02:18:44.071 INFO:teuthology.orchestra.run.trial107.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-13T02:18:44.071 INFO:teuthology.orchestra.run.trial107.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-13T02:18:44.071 INFO:teuthology.orchestra.run.trial107.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-13T02:18:44.071 INFO:teuthology.orchestra.run.trial107.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-13T02:18:44.071 INFO:teuthology.orchestra.run.trial107.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-13T02:18:44.072 INFO:teuthology.orchestra.run.trial107.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-13T02:18:44.072 INFO:teuthology.orchestra.run.trial107.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-13T02:18:44.072 INFO:teuthology.orchestra.run.trial107.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-13T02:18:44.072 INFO:teuthology.orchestra.run.trial107.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-13T02:18:44.072 INFO:teuthology.orchestra.run.trial107.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-13T02:18:44.072 INFO:teuthology.orchestra.run.trial107.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-13T02:18:44.072 INFO:teuthology.orchestra.run.trial107.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-13T02:18:44.072 INFO:teuthology.orchestra.run.trial107.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-13T02:18:44.072 INFO:teuthology.orchestra.run.trial107.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-13T02:18:44.072 INFO:teuthology.orchestra.run.trial107.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-13T02:18:44.072 INFO:teuthology.orchestra.run.trial107.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-13T02:18:44.072 INFO:teuthology.orchestra.run.trial107.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-13T02:18:44.072 INFO:teuthology.orchestra.run.trial107.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-13T02:18:44.073 INFO:teuthology.orchestra.run.trial107.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-13T02:18:44.073 INFO:teuthology.orchestra.run.trial107.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-13T02:18:44.073 INFO:teuthology.orchestra.run.trial107.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2026-02-13T02:18:44.073 INFO:teuthology.orchestra.run.trial107.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-13T02:18:44.073 INFO:teuthology.orchestra.run.trial107.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-13T02:18:44.073 INFO:teuthology.orchestra.run.trial107.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-13T02:18:44.073 INFO:teuthology.orchestra.run.trial107.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-13T02:18:44.073 INFO:teuthology.orchestra.run.trial107.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-13T02:18:44.073 INFO:teuthology.orchestra.run.trial107.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-13T02:18:44.073 INFO:teuthology.orchestra.run.trial107.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-13T02:18:44.073 INFO:teuthology.orchestra.run.trial107.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-13T02:18:44.073 INFO:teuthology.orchestra.run.trial107.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-13T02:18:44.073 INFO:teuthology.orchestra.run.trial107.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-13T02:18:44.073 INFO:teuthology.orchestra.run.trial107.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2026-02-13T02:18:44.074 INFO:teuthology.orchestra.run.trial107.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-13T02:18:44.074 INFO:teuthology.orchestra.run.trial107.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-13T02:18:44.074 INFO:teuthology.orchestra.run.trial107.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-13T02:18:44.074 INFO:teuthology.orchestra.run.trial107.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-13T02:18:44.074 INFO:teuthology.orchestra.run.trial107.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-13T02:18:44.074 INFO:teuthology.orchestra.run.trial107.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-13T02:18:44.074 INFO:teuthology.orchestra.run.trial107.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-13T02:18:44.074 INFO:teuthology.orchestra.run.trial107.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-13T02:18:44.074 INFO:teuthology.orchestra.run.trial107.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-13T02:18:44.074 INFO:teuthology.orchestra.run.trial107.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2026-02-13T02:18:44.074 INFO:teuthology.orchestra.run.trial107.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-13T02:18:44.074 INFO:teuthology.orchestra.run.trial107.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-13T02:18:44.075 INFO:teuthology.orchestra.run.trial107.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-13T02:18:44.075 INFO:teuthology.orchestra.run.trial107.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2026-02-13T02:18:44.075 INFO:teuthology.orchestra.run.trial107.stdout:Installing weak dependencies: 2026-02-13T02:18:44.075 INFO:teuthology.orchestra.run.trial107.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-13T02:18:44.075 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:18:44.075 INFO:teuthology.orchestra.run.trial107.stdout:Transaction Summary 2026-02-13T02:18:44.075 INFO:teuthology.orchestra.run.trial107.stdout:======================================================================================= 2026-02-13T02:18:44.075 INFO:teuthology.orchestra.run.trial107.stdout:Install 162 Packages 2026-02-13T02:18:44.075 INFO:teuthology.orchestra.run.trial107.stdout:Upgrade 11 Packages 2026-02-13T02:18:44.075 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:18:44.076 INFO:teuthology.orchestra.run.trial107.stdout:Total download size: 302 M 2026-02-13T02:18:44.076 INFO:teuthology.orchestra.run.trial107.stdout:Downloading Packages: 2026-02-13T02:18:44.079 INFO:teuthology.orchestra.run.trial027.stdout:(10/173): ceph-osd-classic-20.3.0-5241.g8f8d382 16 MB/s | 16 MB 00:00 2026-02-13T02:18:44.134 INFO:teuthology.orchestra.run.trial027.stdout:(11/173): ceph-selinux-20.3.0-5241.g8f8d3829.el 448 kB/s | 25 kB 00:00 2026-02-13T02:18:44.184 INFO:teuthology.orchestra.run.trial027.stdout:(12/173): ceph-osd-crimson-20.3.0-5241.g8f8d382 22 MB/s | 19 MB 00:00 2026-02-13T02:18:44.244 INFO:teuthology.orchestra.run.trial027.stdout:(13/173): cephfs-mirror-20.3.0-5241.g8f8d3829.e 4.0 MB/s | 243 kB 00:00 2026-02-13T02:18:44.300 INFO:teuthology.orchestra.run.trial027.stdout:(14/173): cephfs-shell-20.3.0-5241.g8f8d3829.el 489 kB/s | 27 kB 00:00 2026-02-13T02:18:44.359 INFO:teuthology.orchestra.run.trial027.stdout:(15/173): libcephfs-devel-20.3.0-5241.g8f8d3829 635 kB/s | 37 kB 00:00 2026-02-13T02:18:44.423 INFO:teuthology.orchestra.run.trial027.stdout:(16/173): ceph-radosgw-20.3.0-5241.g8f8d3829.el 27 MB/s | 13 MB 00:00 2026-02-13T02:18:44.424 INFO:teuthology.orchestra.run.trial027.stdout:(17/173): libcephfs-proxy2-20.3.0-5241.g8f8d382 377 kB/s | 24 kB 00:00 2026-02-13T02:18:44.474 INFO:teuthology.orchestra.run.trial027.stdout:(18/173): libcephfs2-20.3.0-5241.g8f8d3829.el9. 18 MB/s | 909 kB 00:00 2026-02-13T02:18:44.477 INFO:teuthology.orchestra.run.trial027.stdout:(19/173): libcephsqlite-20.3.0-5241.g8f8d3829.e 3.0 MB/s | 159 kB 00:00 2026-02-13T02:18:44.507 INFO:teuthology.orchestra.run.trial027.stdout:(20/173): librados-devel-20.3.0-5241.g8f8d3829. 3.6 MB/s | 123 kB 00:00 2026-02-13T02:18:44.512 INFO:teuthology.orchestra.run.trial027.stdout:(21/173): libradosstriper1-20.3.0-5241.g8f8d382 6.8 MB/s | 248 kB 00:00 2026-02-13T02:18:44.514 INFO:teuthology.orchestra.run.trial107.stdout:(1/173): ceph-20.3.0-5241.g8f8d3829.el9.x86_64. 33 kB/s | 6.5 kB 00:00 2026-02-13T02:18:44.581 INFO:teuthology.orchestra.run.trial027.stdout:(22/173): python3-ceph-argparse-20.3.0-5241.g8f 658 kB/s | 45 kB 00:00 2026-02-13T02:18:44.752 INFO:teuthology.orchestra.run.trial027.stdout:(23/173): librgw2-20.3.0-5241.g8f8d3829.el9.x86 26 MB/s | 6.5 MB 00:00 2026-02-13T02:18:44.777 INFO:teuthology.orchestra.run.trial027.stdout:(24/173): python3-cephfs-20.3.0-5241.g8f8d3829. 8.0 MB/s | 204 kB 00:00 2026-02-13T02:18:44.804 INFO:teuthology.orchestra.run.trial027.stdout:(25/173): python3-rados-20.3.0-5241.g8f8d3829.e 11 MB/s | 324 kB 00:00 2026-02-13T02:18:44.831 INFO:teuthology.orchestra.run.trial027.stdout:(26/173): python3-rbd-20.3.0-5241.g8f8d3829.el9 11 MB/s | 306 kB 00:00 2026-02-13T02:18:44.855 INFO:teuthology.orchestra.run.trial027.stdout:(27/173): python3-rgw-20.3.0-5241.g8f8d3829.el9 4.2 MB/s | 100 kB 00:00 2026-02-13T02:18:44.877 INFO:teuthology.orchestra.run.trial027.stdout:(28/173): rbd-fuse-20.3.0-5241.g8f8d3829.el9.x8 3.7 MB/s | 84 kB 00:00 2026-02-13T02:18:44.894 INFO:teuthology.orchestra.run.trial027.stdout:(29/173): python3-ceph-common-20.3.0-5241.g8f8d 589 kB/s | 185 kB 00:00 2026-02-13T02:18:44.935 INFO:teuthology.orchestra.run.trial027.stdout:(30/173): rbd-nbd-20.3.0-5241.g8f8d3829.el9.x86 4.1 MB/s | 172 kB 00:00 2026-02-13T02:18:44.974 INFO:teuthology.orchestra.run.trial027.stdout:(31/173): ceph-grafana-dashboards-20.3.0-5241.g 1.4 MB/s | 57 kB 00:00 2026-02-13T02:18:44.998 INFO:teuthology.orchestra.run.trial027.stdout:(32/173): rbd-mirror-20.3.0-5241.g8f8d3829.el9. 24 MB/s | 2.9 MB 00:00 2026-02-13T02:18:45.017 INFO:teuthology.orchestra.run.trial027.stdout:(33/173): ceph-mgr-cephadm-20.3.0-5241.g8f8d382 4.2 MB/s | 183 kB 00:00 2026-02-13T02:18:45.507 INFO:teuthology.orchestra.run.trial027.stdout:(34/173): ceph-mgr-diskprediction-local-20.3.0- 15 MB/s | 7.4 MB 00:00 2026-02-13T02:18:45.655 INFO:teuthology.orchestra.run.trial027.stdout:(35/173): ceph-mgr-modules-core-20.3.0-5241.g8f 2.0 MB/s | 297 kB 00:00 2026-02-13T02:18:45.700 INFO:teuthology.orchestra.run.trial027.stdout:(36/173): ceph-mgr-rook-20.3.0-5241.g8f8d3829.e 1.1 MB/s | 50 kB 00:00 2026-02-13T02:18:45.740 INFO:teuthology.orchestra.run.trial027.stdout:(37/173): ceph-prometheus-alerts-20.3.0-5241.g8 441 kB/s | 17 kB 00:00 2026-02-13T02:18:45.846 INFO:teuthology.orchestra.run.trial027.stdout:(38/173): ceph-volume-20.3.0-5241.g8f8d3829.el9 2.7 MB/s | 299 kB 00:00 2026-02-13T02:18:45.907 INFO:teuthology.orchestra.run.trial027.stdout:(39/173): ceph-mgr-dashboard-20.3.0-5241.g8f8d3 16 MB/s | 15 MB 00:00 2026-02-13T02:18:45.933 INFO:teuthology.orchestra.run.trial027.stdout:(40/173): cephfs-top-20.3.0-5241.g8f8d3829.el9. 710 kB/s | 18 kB 00:00 2026-02-13T02:18:45.962 INFO:teuthology.orchestra.run.trial107.stdout:(2/173): ceph-fuse-20.3.0-5241.g8f8d3829.el9.x8 682 kB/s | 987 kB 00:01 2026-02-13T02:18:46.036 INFO:teuthology.orchestra.run.trial027.stdout:(41/173): cephadm-20.3.0-5241.g8f8d3829.el9.noa 5.4 MB/s | 1.0 MB 00:00 2026-02-13T02:18:46.083 INFO:teuthology.orchestra.run.trial107.stdout:(3/173): ceph-immutable-object-cache-20.3.0-524 1.2 MB/s | 147 kB 00:00 2026-02-13T02:18:46.473 INFO:teuthology.orchestra.run.trial027.stdout:(42/173): cryptsetup-2.8.1-2.el9.x86_64.rpm 649 kB/s | 350 kB 00:00 2026-02-13T02:18:46.715 INFO:teuthology.orchestra.run.trial027.stdout:(43/173): hwloc-libs-2.4.1-6.el9.x86_64.rpm 3.1 MB/s | 2.1 MB 00:00 2026-02-13T02:18:46.727 INFO:teuthology.orchestra.run.trial107.stdout:(4/173): ceph-common-20.3.0-5241.g8f8d3829.el9. 10 MB/s | 24 MB 00:02 2026-02-13T02:18:46.762 INFO:teuthology.orchestra.run.trial107.stdout:(5/173): ceph-mgr-20.3.0-5241.g8f8d3829.el9.x86 27 MB/s | 940 kB 00:00 2026-02-13T02:18:46.809 INFO:teuthology.orchestra.run.trial027.stdout:(44/173): ledmon-libs-1.1.0-3.el9.x86_64.rpm 120 kB/s | 40 kB 00:00 2026-02-13T02:18:46.924 INFO:teuthology.orchestra.run.trial107.stdout:(6/173): ceph-mon-20.3.0-5241.g8f8d3829.el9.x86 31 MB/s | 5.0 MB 00:00 2026-02-13T02:18:46.938 INFO:teuthology.orchestra.run.trial107.stdout:(7/173): ceph-base-20.3.0-5241.g8f8d3829.el9.x8 2.0 MB/s | 5.2 MB 00:02 2026-02-13T02:18:46.945 INFO:teuthology.orchestra.run.trial107.stdout:(8/173): ceph-osd-20.3.0-5241.g8f8d3829.el9.x86 7.7 MB/s | 166 kB 00:00 2026-02-13T02:18:46.985 INFO:teuthology.orchestra.run.trial027.stdout:(45/173): libconfig-1.7.2-9.el9.x86_64.rpm 409 kB/s | 72 kB 00:00 2026-02-13T02:18:47.195 INFO:teuthology.orchestra.run.trial027.stdout:(46/173): libatomic-11.5.0-14.el9.x86_64.rpm 47 kB/s | 23 kB 00:00 2026-02-13T02:18:47.399 INFO:teuthology.orchestra.run.trial027.stdout:(47/173): ceph-test-20.3.0-5241.g8f8d3829.el9.x 24 MB/s | 78 MB 00:03 2026-02-13T02:18:47.437 INFO:teuthology.orchestra.run.trial107.stdout:(9/173): ceph-osd-crimson-20.3.0-5241.g8f8d3829 39 MB/s | 19 MB 00:00 2026-02-13T02:18:47.712 INFO:teuthology.orchestra.run.trial107.stdout:(10/173): ceph-radosgw-20.3.0-5241.g8f8d3829.el 48 MB/s | 13 MB 00:00 2026-02-13T02:18:47.737 INFO:teuthology.orchestra.run.trial107.stdout:(11/173): ceph-mds-20.3.0-5241.g8f8d3829.el9.x8 1.4 MB/s | 2.4 MB 00:01 2026-02-13T02:18:47.737 INFO:teuthology.orchestra.run.trial107.stdout:(12/173): ceph-selinux-20.3.0-5241.g8f8d3829.el 1.0 MB/s | 25 kB 00:00 2026-02-13T02:18:47.764 INFO:teuthology.orchestra.run.trial107.stdout:(13/173): cephfs-mirror-20.3.0-5241.g8f8d3829.e 8.8 MB/s | 243 kB 00:00 2026-02-13T02:18:47.771 INFO:teuthology.orchestra.run.trial027.stdout:(48/173): libgfortran-11.5.0-14.el9.x86_64.rpm 1.0 MB/s | 794 kB 00:00 2026-02-13T02:18:47.777 INFO:teuthology.orchestra.run.trial107.stdout:(14/173): cephfs-shell-20.3.0-5241.g8f8d3829.el 2.2 MB/s | 27 kB 00:00 2026-02-13T02:18:47.788 INFO:teuthology.orchestra.run.trial107.stdout:(15/173): libcephfs-devel-20.3.0-5241.g8f8d3829 3.2 MB/s | 37 kB 00:00 2026-02-13T02:18:47.796 INFO:teuthology.orchestra.run.trial027.stdout:(49/173): libquadmath-11.5.0-14.el9.x86_64.rpm 307 kB/s | 184 kB 00:00 2026-02-13T02:18:47.801 INFO:teuthology.orchestra.run.trial107.stdout:(16/173): libcephfs-proxy2-20.3.0-5241.g8f8d382 1.8 MB/s | 24 kB 00:00 2026-02-13T02:18:47.822 INFO:teuthology.orchestra.run.trial107.stdout:(17/173): libcephfs2-20.3.0-5241.g8f8d3829.el9. 44 MB/s | 909 kB 00:00 2026-02-13T02:18:47.834 INFO:teuthology.orchestra.run.trial107.stdout:(18/173): libcephsqlite-20.3.0-5241.g8f8d3829.e 13 MB/s | 159 kB 00:00 2026-02-13T02:18:47.845 INFO:teuthology.orchestra.run.trial107.stdout:(19/173): librados-devel-20.3.0-5241.g8f8d3829. 10 MB/s | 123 kB 00:00 2026-02-13T02:18:47.857 INFO:teuthology.orchestra.run.trial107.stdout:(20/173): libradosstriper1-20.3.0-5241.g8f8d382 20 MB/s | 248 kB 00:00 2026-02-13T02:18:47.925 INFO:teuthology.orchestra.run.trial027.stdout:(50/173): mailcap-2.1.49-5.el9.noarch.rpm 216 kB/s | 33 kB 00:00 2026-02-13T02:18:47.931 INFO:teuthology.orchestra.run.trial027.stdout:(51/173): lksctp-tools-1.0.19-2.el9.x86_64.rpm 177 kB/s | 94 kB 00:00 2026-02-13T02:18:47.963 INFO:teuthology.orchestra.run.trial027.stdout:(52/173): pciutils-3.7.0-7.el9.x86_64.rpm 561 kB/s | 93 kB 00:00 2026-02-13T02:18:47.985 INFO:teuthology.orchestra.run.trial107.stdout:(21/173): librgw2-20.3.0-5241.g8f8d3829.el9.x86 51 MB/s | 6.5 MB 00:00 2026-02-13T02:18:47.997 INFO:teuthology.orchestra.run.trial107.stdout:(22/173): python3-ceph-argparse-20.3.0-5241.g8f 3.7 MB/s | 45 kB 00:00 2026-02-13T02:18:48.009 INFO:teuthology.orchestra.run.trial107.stdout:(23/173): python3-ceph-common-20.3.0-5241.g8f8d 15 MB/s | 185 kB 00:00 2026-02-13T02:18:48.021 INFO:teuthology.orchestra.run.trial107.stdout:(24/173): python3-cephfs-20.3.0-5241.g8f8d3829. 17 MB/s | 204 kB 00:00 2026-02-13T02:18:48.033 INFO:teuthology.orchestra.run.trial107.stdout:(25/173): python3-rados-20.3.0-5241.g8f8d3829.e 26 MB/s | 324 kB 00:00 2026-02-13T02:18:48.045 INFO:teuthology.orchestra.run.trial107.stdout:(26/173): python3-rbd-20.3.0-5241.g8f8d3829.el9 25 MB/s | 306 kB 00:00 2026-02-13T02:18:48.057 INFO:teuthology.orchestra.run.trial107.stdout:(27/173): python3-rgw-20.3.0-5241.g8f8d3829.el9 8.5 MB/s | 100 kB 00:00 2026-02-13T02:18:48.069 INFO:teuthology.orchestra.run.trial107.stdout:(28/173): rbd-fuse-20.3.0-5241.g8f8d3829.el9.x8 7.1 MB/s | 84 kB 00:00 2026-02-13T02:18:48.126 INFO:teuthology.orchestra.run.trial107.stdout:(29/173): rbd-mirror-20.3.0-5241.g8f8d3829.el9. 50 MB/s | 2.9 MB 00:00 2026-02-13T02:18:48.138 INFO:teuthology.orchestra.run.trial107.stdout:(30/173): rbd-nbd-20.3.0-5241.g8f8d3829.el9.x86 14 MB/s | 172 kB 00:00 2026-02-13T02:18:48.150 INFO:teuthology.orchestra.run.trial107.stdout:(31/173): ceph-grafana-dashboards-20.3.0-5241.g 5.0 MB/s | 57 kB 00:00 2026-02-13T02:18:48.161 INFO:teuthology.orchestra.run.trial107.stdout:(32/173): ceph-mgr-cephadm-20.3.0-5241.g8f8d382 15 MB/s | 183 kB 00:00 2026-02-13T02:18:48.178 INFO:teuthology.orchestra.run.trial027.stdout:(53/173): python3-ply-3.11-14.el9.noarch.rpm 493 kB/s | 106 kB 00:00 2026-02-13T02:18:48.243 INFO:teuthology.orchestra.run.trial027.stdout:(54/173): python3-cffi-1.14.5-5.el9.x86_64.rpm 797 kB/s | 253 kB 00:00 2026-02-13T02:18:48.290 INFO:teuthology.orchestra.run.trial027.stdout:(55/173): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2026-02-13T02:18:48.410 INFO:teuthology.orchestra.run.trial027.stdout:(56/173): python3-requests-2.25.1-10.el9.noarch 757 kB/s | 126 kB 00:00 2026-02-13T02:18:48.441 INFO:teuthology.orchestra.run.trial027.stdout:(57/173): python3-urllib3-1.26.5-7.el9.noarch.r 1.4 MB/s | 218 kB 00:00 2026-02-13T02:18:48.459 INFO:teuthology.orchestra.run.trial107.stdout:(33/173): ceph-mgr-dashboard-20.3.0-5241.g8f8d3 50 MB/s | 15 MB 00:00 2026-02-13T02:18:48.505 INFO:teuthology.orchestra.run.trial027.stdout:(58/173): python3-cryptography-36.0.1-5.el9.x86 2.2 MB/s | 1.2 MB 00:00 2026-02-13T02:18:48.608 INFO:teuthology.orchestra.run.trial107.stdout:(34/173): ceph-mgr-diskprediction-local-20.3.0- 50 MB/s | 7.4 MB 00:00 2026-02-13T02:18:48.621 INFO:teuthology.orchestra.run.trial107.stdout:(35/173): ceph-mgr-modules-core-20.3.0-5241.g8f 23 MB/s | 297 kB 00:00 2026-02-13T02:18:48.633 INFO:teuthology.orchestra.run.trial107.stdout:(36/173): ceph-mgr-rook-20.3.0-5241.g8f8d3829.e 4.4 MB/s | 50 kB 00:00 2026-02-13T02:18:48.645 INFO:teuthology.orchestra.run.trial107.stdout:(37/173): ceph-prometheus-alerts-20.3.0-5241.g8 1.4 MB/s | 17 kB 00:00 2026-02-13T02:18:48.657 INFO:teuthology.orchestra.run.trial107.stdout:(38/173): ceph-volume-20.3.0-5241.g8f8d3829.el9 24 MB/s | 299 kB 00:00 2026-02-13T02:18:48.679 INFO:teuthology.orchestra.run.trial107.stdout:(39/173): cephadm-20.3.0-5241.g8f8d3829.el9.noa 47 MB/s | 1.0 MB 00:00 2026-02-13T02:18:48.691 INFO:teuthology.orchestra.run.trial107.stdout:(40/173): cephfs-top-20.3.0-5241.g8f8d3829.el9. 1.6 MB/s | 18 kB 00:00 2026-02-13T02:18:48.727 INFO:teuthology.orchestra.run.trial027.stdout:(59/173): unzip-6.0-59.el9.x86_64.rpm 574 kB/s | 182 kB 00:00 2026-02-13T02:18:48.802 INFO:teuthology.orchestra.run.trial027.stdout:(60/173): zip-3.0-35.el9.x86_64.rpm 736 kB/s | 266 kB 00:00 2026-02-13T02:18:49.269 INFO:teuthology.orchestra.run.trial107.stdout:(41/173): ceph-osd-classic-20.3.0-5241.g8f8d382 6.7 MB/s | 16 MB 00:02 2026-02-13T02:18:49.358 INFO:teuthology.orchestra.run.trial027.stdout:(61/173): boost-program-options-1.75.0-13.el9.x 187 kB/s | 104 kB 00:00 2026-02-13T02:18:49.518 INFO:teuthology.orchestra.run.trial027.stdout:(62/173): elfutils-libelf-devel-0.194-1.el9.x86 496 kB/s | 79 kB 00:00 2026-02-13T02:18:49.528 INFO:teuthology.orchestra.run.trial107.stdout:(42/173): cryptsetup-2.8.1-2.el9.x86_64.rpm 419 kB/s | 350 kB 00:00 2026-02-13T02:18:49.627 INFO:teuthology.orchestra.run.trial107.stdout:(43/173): ledmon-libs-1.1.0-3.el9.x86_64.rpm 408 kB/s | 40 kB 00:00 2026-02-13T02:18:49.707 INFO:teuthology.orchestra.run.trial027.stdout:(63/173): automake-1.16.2-8.el9.noarch.rpm 573 kB/s | 689 kB 00:01 2026-02-13T02:18:49.726 INFO:teuthology.orchestra.run.trial107.stdout:(44/173): libatomic-11.5.0-14.el9.x86_64.rpm 231 kB/s | 23 kB 00:00 2026-02-13T02:18:49.790 INFO:teuthology.orchestra.run.trial027.stdout:(64/173): flexiblas-3.0.4-8.el9.x86_64.rpm 395 kB/s | 33 kB 00:00 2026-02-13T02:18:49.904 INFO:teuthology.orchestra.run.trial107.stdout:(45/173): libconfig-1.7.2-9.el9.x86_64.rpm 406 kB/s | 72 kB 00:00 2026-02-13T02:18:49.912 INFO:teuthology.orchestra.run.trial027.stdout:(65/173): flex-2.6.4-9.el9.x86_64.rpm 796 kB/s | 313 kB 00:00 2026-02-13T02:18:49.994 INFO:teuthology.orchestra.run.trial027.stdout:(66/173): flexiblas-openblas-openmp-3.0.4-8.el9 221 kB/s | 18 kB 00:00 2026-02-13T02:18:50.077 INFO:teuthology.orchestra.run.trial027.stdout:(67/173): libaio-devel-0.3.111-13.el9.x86_64.rp 136 kB/s | 11 kB 00:00 2026-02-13T02:18:50.218 INFO:teuthology.orchestra.run.trial027.stdout:(68/173): bison-3.7.4-5.el9.x86_64.rpm 632 kB/s | 943 kB 00:01 2026-02-13T02:18:50.314 INFO:teuthology.orchestra.run.trial027.stdout:(69/173): libnbd-1.20.3-4.el9.x86_64.rpm 689 kB/s | 164 kB 00:00 2026-02-13T02:18:50.397 INFO:teuthology.orchestra.run.trial027.stdout:(70/173): librabbitmq-0.11.0-7.el9.x86_64.rpm 550 kB/s | 45 kB 00:00 2026-02-13T02:18:50.456 INFO:teuthology.orchestra.run.trial027.stdout:(71/173): libpmemobj-1.12.1-1.el9.x86_64.rpm 673 kB/s | 160 kB 00:00 2026-02-13T02:18:50.772 INFO:teuthology.orchestra.run.trial027.stdout:(72/173): libstoragemgmt-1.10.1-1.el9.x86_64.rp 779 kB/s | 246 kB 00:00 2026-02-13T02:18:50.867 INFO:teuthology.orchestra.run.trial107.stdout:(46/173): libgfortran-11.5.0-14.el9.x86_64.rpm 825 kB/s | 794 kB 00:00 2026-02-13T02:18:51.123 INFO:teuthology.orchestra.run.trial107.stdout:(47/173): libquadmath-11.5.0-14.el9.x86_64.rpm 721 kB/s | 184 kB 00:00 2026-02-13T02:18:51.180 INFO:teuthology.orchestra.run.trial027.stdout:(73/173): librdkafka-1.6.1-102.el9.x86_64.rpm 846 kB/s | 662 kB 00:00 2026-02-13T02:18:51.262 INFO:teuthology.orchestra.run.trial027.stdout:(74/173): libuuid-devel-2.37.4-21.el9.x86_64.rp 297 kB/s | 25 kB 00:00 2026-02-13T02:18:51.300 INFO:teuthology.orchestra.run.trial107.stdout:(48/173): lksctp-tools-1.0.19-2.el9.x86_64.rpm 530 kB/s | 94 kB 00:00 2026-02-13T02:18:51.400 INFO:teuthology.orchestra.run.trial107.stdout:(49/173): mailcap-2.1.49-5.el9.noarch.rpm 335 kB/s | 33 kB 00:00 2026-02-13T02:18:51.478 INFO:teuthology.orchestra.run.trial027.stdout:(75/173): libtool-2.4.6-46.el9.x86_64.rpm 823 kB/s | 581 kB 00:00 2026-02-13T02:18:51.560 INFO:teuthology.orchestra.run.trial027.stdout:(76/173): libzstd-devel-1.5.5-1.el9.x86_64.rpm 614 kB/s | 51 kB 00:00 2026-02-13T02:18:51.577 INFO:teuthology.orchestra.run.trial107.stdout:(50/173): pciutils-3.7.0-7.el9.x86_64.rpm 524 kB/s | 93 kB 00:00 2026-02-13T02:18:51.579 INFO:teuthology.orchestra.run.trial027.stdout:(77/173): libxslt-1.1.34-12.el9.x86_64.rpm 735 kB/s | 233 kB 00:00 2026-02-13T02:18:51.818 INFO:teuthology.orchestra.run.trial027.stdout:(78/173): lua-5.4.4-4.el9.x86_64.rpm 791 kB/s | 188 kB 00:00 2026-02-13T02:18:51.900 INFO:teuthology.orchestra.run.trial027.stdout:(79/173): openblas-0.3.29-1.el9.x86_64.rpm 511 kB/s | 42 kB 00:00 2026-02-13T02:18:51.913 INFO:teuthology.orchestra.run.trial107.stdout:(51/173): python3-cffi-1.14.5-5.el9.x86_64.rpm 755 kB/s | 253 kB 00:00 2026-02-13T02:18:51.955 INFO:teuthology.orchestra.run.trial027.stdout:(80/173): lttng-ust-2.12.0-6.el9.x86_64.rpm 740 kB/s | 292 kB 00:00 2026-02-13T02:18:52.152 INFO:teuthology.orchestra.run.trial107.stdout:(52/173): hwloc-libs-2.4.1-6.el9.x86_64.rpm 745 kB/s | 2.1 MB 00:02 2026-02-13T02:18:52.333 INFO:teuthology.orchestra.run.trial107.stdout:(53/173): python3-ply-3.11-14.el9.noarch.rpm 589 kB/s | 106 kB 00:00 2026-02-13T02:18:52.491 INFO:teuthology.orchestra.run.trial107.stdout:(54/173): ceph-test-20.3.0-5241.g8f8d3829.el9.x 16 MB/s | 78 MB 00:04 2026-02-13T02:18:52.550 INFO:teuthology.orchestra.run.trial107.stdout:(55/173): python3-pycparser-2.20-6.el9.noarch.r 622 kB/s | 135 kB 00:00 2026-02-13T02:18:52.884 INFO:teuthology.orchestra.run.trial107.stdout:(56/173): python3-urllib3-1.26.5-7.el9.noarch.r 652 kB/s | 218 kB 00:00 2026-02-13T02:18:53.065 INFO:teuthology.orchestra.run.trial107.stdout:(57/173): python3-requests-2.25.1-10.el9.noarch 220 kB/s | 126 kB 00:00 2026-02-13T02:18:53.140 INFO:teuthology.orchestra.run.trial107.stdout:(58/173): unzip-6.0-59.el9.x86_64.rpm 710 kB/s | 182 kB 00:00 2026-02-13T02:18:53.250 INFO:teuthology.orchestra.run.trial027.stdout:(81/173): flexiblas-netlib-3.0.4-8.el9.x86_64.r 885 kB/s | 3.0 MB 00:03 2026-02-13T02:18:53.333 INFO:teuthology.orchestra.run.trial027.stdout:(82/173): perl-Thread-Queue-3.14-460.el9.noarch 268 kB/s | 22 kB 00:00 2026-02-13T02:18:53.403 INFO:teuthology.orchestra.run.trial107.stdout:(59/173): zip-3.0-35.el9.x86_64.rpm 787 kB/s | 266 kB 00:00 2026-02-13T02:18:53.417 INFO:teuthology.orchestra.run.trial027.stdout:(83/173): perl-threads-2.25-460.el9.x86_64.rpm 705 kB/s | 58 kB 00:00 2026-02-13T02:18:53.468 INFO:teuthology.orchestra.run.trial107.stdout:(60/173): python3-cryptography-36.0.1-5.el9.x86 820 kB/s | 1.2 MB 00:01 2026-02-13T02:18:53.499 INFO:teuthology.orchestra.run.trial027.stdout:(84/173): perl-threads-shared-1.61-460.el9.x86_ 539 kB/s | 45 kB 00:00 2026-02-13T02:18:53.793 INFO:teuthology.orchestra.run.trial107.stdout:(61/173): automake-1.16.2-8.el9.noarch.rpm 1.0 MB/s | 689 kB 00:00 2026-02-13T02:18:53.880 INFO:teuthology.orchestra.run.trial107.stdout:(62/173): elfutils-libelf-devel-0.194-1.el9.x86 910 kB/s | 79 kB 00:00 2026-02-13T02:18:53.881 INFO:teuthology.orchestra.run.trial107.stdout:(63/173): boost-program-options-1.75.0-13.el9.x 252 kB/s | 104 kB 00:00 2026-02-13T02:18:53.978 INFO:teuthology.orchestra.run.trial107.stdout:(64/173): flexiblas-3.0.4-8.el9.x86_64.rpm 337 kB/s | 33 kB 00:00 2026-02-13T02:18:53.987 INFO:teuthology.orchestra.run.trial107.stdout:(65/173): flex-2.6.4-9.el9.x86_64.rpm 2.9 MB/s | 313 kB 00:00 2026-02-13T02:18:54.069 INFO:teuthology.orchestra.run.trial107.stdout:(66/173): flexiblas-openblas-openmp-3.0.4-8.el9 222 kB/s | 18 kB 00:00 2026-02-13T02:18:54.081 INFO:teuthology.orchestra.run.trial107.stdout:(67/173): bison-3.7.4-5.el9.x86_64.rpm 1.4 MB/s | 943 kB 00:00 2026-02-13T02:18:54.151 INFO:teuthology.orchestra.run.trial107.stdout:(68/173): libaio-devel-0.3.111-13.el9.x86_64.rp 136 kB/s | 11 kB 00:00 2026-02-13T02:18:54.172 INFO:teuthology.orchestra.run.trial107.stdout:(69/173): libnbd-1.20.3-4.el9.x86_64.rpm 1.8 MB/s | 164 kB 00:00 2026-02-13T02:18:54.245 INFO:teuthology.orchestra.run.trial107.stdout:(70/173): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2026-02-13T02:18:54.255 INFO:teuthology.orchestra.run.trial107.stdout:(71/173): librabbitmq-0.11.0-7.el9.x86_64.rpm 544 kB/s | 45 kB 00:00 2026-02-13T02:18:54.350 INFO:teuthology.orchestra.run.trial107.stdout:(72/173): libstoragemgmt-1.10.1-1.el9.x86_64.rp 2.5 MB/s | 246 kB 00:00 2026-02-13T02:18:54.381 INFO:teuthology.orchestra.run.trial107.stdout:(73/173): librdkafka-1.6.1-102.el9.x86_64.rpm 4.8 MB/s | 662 kB 00:00 2026-02-13T02:18:54.465 INFO:teuthology.orchestra.run.trial107.stdout:(74/173): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.1 MB/s | 3.0 MB 00:00 2026-02-13T02:18:54.465 INFO:teuthology.orchestra.run.trial107.stdout:(75/173): libuuid-devel-2.37.4-21.el9.x86_64.rp 290 kB/s | 25 kB 00:00 2026-02-13T02:18:54.466 INFO:teuthology.orchestra.run.trial107.stdout:(76/173): libtool-2.4.6-46.el9.x86_64.rpm 4.9 MB/s | 581 kB 00:00 2026-02-13T02:18:54.549 INFO:teuthology.orchestra.run.trial107.stdout:(77/173): libxslt-1.1.34-12.el9.x86_64.rpm 2.7 MB/s | 233 kB 00:00 2026-02-13T02:18:54.550 INFO:teuthology.orchestra.run.trial107.stdout:(78/173): libzstd-devel-1.5.5-1.el9.x86_64.rpm 599 kB/s | 51 kB 00:00 2026-02-13T02:18:54.564 INFO:teuthology.orchestra.run.trial107.stdout:(79/173): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.9 MB/s | 292 kB 00:00 2026-02-13T02:18:54.633 INFO:teuthology.orchestra.run.trial107.stdout:(80/173): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2026-02-13T02:18:54.634 INFO:teuthology.orchestra.run.trial107.stdout:(81/173): openblas-0.3.29-1.el9.x86_64.rpm 501 kB/s | 42 kB 00:00 2026-02-13T02:18:54.716 INFO:teuthology.orchestra.run.trial107.stdout:(82/173): perl-Thread-Queue-3.14-460.el9.noarch 271 kB/s | 22 kB 00:00 2026-02-13T02:18:54.779 INFO:teuthology.orchestra.run.trial028.stdout:CentOS Stream 9 - AppStream 978 kB/s | 27 MB 00:27 2026-02-13T02:18:54.801 INFO:teuthology.orchestra.run.trial107.stdout:(83/173): perl-threads-2.25-460.el9.x86_64.rpm 689 kB/s | 58 kB 00:00 2026-02-13T02:18:54.838 INFO:teuthology.orchestra.run.trial107.stdout:(84/173): openssl-devel-3.5.5-1.el9.x86_64.rpm 23 MB/s | 4.8 MB 00:00 2026-02-13T02:18:54.873 INFO:teuthology.orchestra.run.trial107.stdout:(85/173): openblas-openmp-0.3.29-1.el9.x86_64.r 17 MB/s | 5.3 MB 00:00 2026-02-13T02:18:54.885 INFO:teuthology.orchestra.run.trial107.stdout:(86/173): perl-threads-shared-1.61-460.el9.x86_ 532 kB/s | 45 kB 00:00 2026-02-13T02:18:54.933 INFO:teuthology.orchestra.run.trial107.stdout:(87/173): postgresql-13.23-1.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2026-02-13T02:18:54.972 INFO:teuthology.orchestra.run.trial107.stdout:(88/173): postgresql-contrib-13.23-1.el9.x86_64 8.9 MB/s | 903 kB 00:00 2026-02-13T02:18:54.976 INFO:teuthology.orchestra.run.trial107.stdout:(89/173): postgresql-private-libs-13.23-1.el9.x 1.5 MB/s | 138 kB 00:00 2026-02-13T02:18:55.064 INFO:teuthology.orchestra.run.trial107.stdout:(90/173): protobuf-3.14.0-17.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2026-02-13T02:18:55.149 INFO:teuthology.orchestra.run.trial107.stdout:(91/173): postgresql-server-13.23-1.el9.x86_64. 27 MB/s | 5.8 MB 00:00 2026-02-13T02:18:55.158 INFO:teuthology.orchestra.run.trial107.stdout:(92/173): python3-colorama-0.4.6-3.el9.noarch.r 647 kB/s | 61 kB 00:00 2026-02-13T02:18:55.234 INFO:teuthology.orchestra.run.trial107.stdout:(93/173): python3-devel-3.9.25-2.el9.x86_64.rpm 2.8 MB/s | 244 kB 00:00 2026-02-13T02:18:55.242 INFO:teuthology.orchestra.run.trial107.stdout:(94/173): python3-jinja2-2.11.3-8.el9.noarch.rp 2.9 MB/s | 249 kB 00:00 2026-02-13T02:18:55.316 INFO:teuthology.orchestra.run.trial107.stdout:(95/173): python3-jmespath-1.0.1-1.el9.noarch.r 577 kB/s | 48 kB 00:00 2026-02-13T02:18:55.328 INFO:teuthology.orchestra.run.trial107.stdout:(96/173): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2026-02-13T02:18:55.329 INFO:teuthology.orchestra.run.trial107.stdout:(97/173): python3-libstoragemgmt-1.10.1-1.el9.x 2.0 MB/s | 177 kB 00:00 2026-02-13T02:18:55.389 INFO:teuthology.orchestra.run.trial027.stdout:(85/173): postgresql-13.23-1.el9.x86_64.rpm 880 kB/s | 1.6 MB 00:01 2026-02-13T02:18:55.398 INFO:teuthology.orchestra.run.trial107.stdout:(98/173): python3-markupsafe-1.1.1-12.el9.x86_6 423 kB/s | 35 kB 00:00 2026-02-13T02:18:55.416 INFO:teuthology.orchestra.run.trial107.stdout:(99/173): python3-numpy-f2py-1.23.5-2.el9.x86_6 5.0 MB/s | 442 kB 00:00 2026-02-13T02:18:55.481 INFO:teuthology.orchestra.run.trial107.stdout:(100/173): python3-packaging-20.9-5.el9.noarch. 937 kB/s | 77 kB 00:00 2026-02-13T02:18:55.502 INFO:teuthology.orchestra.run.trial107.stdout:(101/173): python3-protobuf-3.14.0-17.el9.noarc 3.1 MB/s | 267 kB 00:00 2026-02-13T02:18:55.531 INFO:teuthology.orchestra.run.trial107.stdout:(102/173): python3-numpy-1.23.5-2.el9.x86_64.rp 30 MB/s | 6.1 MB 00:00 2026-02-13T02:18:55.565 INFO:teuthology.orchestra.run.trial107.stdout:(103/173): python3-pyasn1-0.4.8-6.el9.noarch.rp 1.8 MB/s | 159 kB 00:00 2026-02-13T02:18:55.587 INFO:teuthology.orchestra.run.trial107.stdout:(104/173): python3-pyasn1-modules-0.4.8-6.el9.n 3.2 MB/s | 279 kB 00:00 2026-02-13T02:18:55.614 INFO:teuthology.orchestra.run.trial107.stdout:(105/173): python3-requests-oauthlib-1.3.0-12.e 644 kB/s | 54 kB 00:00 2026-02-13T02:18:55.670 INFO:teuthology.orchestra.run.trial107.stdout:(106/173): python3-toml-0.10.2-6.el9.noarch.rpm 502 kB/s | 42 kB 00:00 2026-02-13T02:18:55.697 INFO:teuthology.orchestra.run.trial107.stdout:(107/173): python3-wcwidth-0.2.5-8.el9.noarch.r 533 kB/s | 44 kB 00:00 2026-02-13T02:18:55.754 INFO:teuthology.orchestra.run.trial107.stdout:(108/173): qatlib-25.08.0-2.el9.x86_64.rpm 2.8 MB/s | 240 kB 00:00 2026-02-13T02:18:55.780 INFO:teuthology.orchestra.run.trial107.stdout:(109/173): qatlib-service-25.08.0-2.el9.x86_64. 451 kB/s | 37 kB 00:00 2026-02-13T02:18:55.837 INFO:teuthology.orchestra.run.trial107.stdout:(110/173): qatzip-libs-1.3.1-1.el9.x86_64.rpm 798 kB/s | 66 kB 00:00 2026-02-13T02:18:55.863 INFO:teuthology.orchestra.run.trial107.stdout:(111/173): socat-1.7.4.1-8.el9.x86_64.rpm 3.5 MB/s | 303 kB 00:00 2026-02-13T02:18:55.920 INFO:teuthology.orchestra.run.trial107.stdout:(112/173): uuid-1.6.2-55.el9.x86_64.rpm 700 kB/s | 58 kB 00:00 2026-02-13T02:18:55.946 INFO:teuthology.orchestra.run.trial107.stdout:(113/173): xfsprogs-devel-6.4.0-7.el9.x86_64.rp 1.1 MB/s | 96 kB 00:00 2026-02-13T02:18:56.003 INFO:teuthology.orchestra.run.trial107.stdout:(114/173): xmlstarlet-1.6.1-20.el9.x86_64.rpm 768 kB/s | 64 kB 00:00 2026-02-13T02:18:56.029 INFO:teuthology.orchestra.run.trial107.stdout:(115/173): zlib-devel-1.2.11-41.el9.x86_64.rpm 543 kB/s | 45 kB 00:00 2026-02-13T02:18:56.176 INFO:teuthology.orchestra.run.trial107.stdout:(116/173): python3-scipy-1.9.3-2.el9.x86_64.rpm 31 MB/s | 19 MB 00:00 2026-02-13T02:18:56.222 INFO:teuthology.orchestra.run.trial107.stdout:(117/173): abseil-cpp-20211102.0-4.el9.x86_64.r 12 MB/s | 551 kB 00:00 2026-02-13T02:18:56.227 INFO:teuthology.orchestra.run.trial107.stdout:(118/173): gperftools-libs-2.9.1-3.el9.x86_64.r 70 MB/s | 308 kB 00:00 2026-02-13T02:18:56.230 INFO:teuthology.orchestra.run.trial107.stdout:(119/173): grpc-data-1.46.7-10.el9.noarch.rpm 5.7 MB/s | 19 kB 00:00 2026-02-13T02:18:56.249 INFO:teuthology.orchestra.run.trial107.stdout:(120/173): libarrow-9.0.0-15.el9.x86_64.rpm 234 MB/s | 4.4 MB 00:00 2026-02-13T02:18:56.256 INFO:teuthology.orchestra.run.trial107.stdout:(121/173): libarrow-doc-9.0.0-15.el9.noarch.rpm 3.7 MB/s | 25 kB 00:00 2026-02-13T02:18:56.259 INFO:teuthology.orchestra.run.trial107.stdout:(122/173): liboath-2.6.12-1.el9.x86_64.rpm 14 MB/s | 49 kB 00:00 2026-02-13T02:18:56.263 INFO:teuthology.orchestra.run.trial107.stdout:(123/173): libunwind-1.6.2-1.el9.x86_64.rpm 18 MB/s | 67 kB 00:00 2026-02-13T02:18:56.267 INFO:teuthology.orchestra.run.trial107.stdout:(124/173): luarocks-3.9.2-5.el9.noarch.rpm 39 MB/s | 151 kB 00:00 2026-02-13T02:18:56.273 INFO:teuthology.orchestra.run.trial107.stdout:(125/173): parquet-libs-9.0.0-15.el9.x86_64.rpm 150 MB/s | 838 kB 00:00 2026-02-13T02:18:56.277 INFO:teuthology.orchestra.run.trial107.stdout:(126/173): python3-asyncssh-2.13.2-5.el9.noarch 116 MB/s | 548 kB 00:00 2026-02-13T02:18:56.281 INFO:teuthology.orchestra.run.trial107.stdout:(127/173): python3-autocommand-2.2.2-8.el9.noar 8.4 MB/s | 29 kB 00:00 2026-02-13T02:18:56.288 INFO:teuthology.orchestra.run.trial107.stdout:(128/173): python3-backports-tarfile-1.2.0-1.el 7.9 MB/s | 60 kB 00:00 2026-02-13T02:18:56.292 INFO:teuthology.orchestra.run.trial107.stdout:(129/173): python3-bcrypt-3.2.2-1.el9.x86_64.rp 13 MB/s | 43 kB 00:00 2026-02-13T02:18:56.296 INFO:teuthology.orchestra.run.trial107.stdout:(130/173): python3-cachetools-4.2.4-1.el9.noarc 8.8 MB/s | 32 kB 00:00 2026-02-13T02:18:56.299 INFO:teuthology.orchestra.run.trial107.stdout:(131/173): python3-certifi-2023.05.07-4.el9.noa 4.2 MB/s | 14 kB 00:00 2026-02-13T02:18:56.303 INFO:teuthology.orchestra.run.trial107.stdout:(132/173): python3-cheroot-10.0.1-4.el9.noarch. 44 MB/s | 173 kB 00:00 2026-02-13T02:18:56.307 INFO:teuthology.orchestra.run.trial107.stdout:(133/173): python3-cherrypy-18.6.1-2.el9.noarch 83 MB/s | 358 kB 00:00 2026-02-13T02:18:56.313 INFO:teuthology.orchestra.run.trial107.stdout:(134/173): python3-cmd2-2.4.2-2.el9.noarch.rpm 56 MB/s | 303 kB 00:00 2026-02-13T02:18:56.317 INFO:teuthology.orchestra.run.trial107.stdout:(135/173): python3-google-auth-2.45.0-1.el9.noa 62 MB/s | 254 kB 00:00 2026-02-13T02:18:56.325 INFO:teuthology.orchestra.run.trial107.stdout:(136/173): python3-grpcio-1.46.7-10.el9.x86_64. 238 MB/s | 2.0 MB 00:00 2026-02-13T02:18:56.329 INFO:teuthology.orchestra.run.trial107.stdout:(137/173): python3-grpcio-tools-1.46.7-10.el9.x 38 MB/s | 144 kB 00:00 2026-02-13T02:18:56.333 INFO:teuthology.orchestra.run.trial107.stdout:(138/173): python3-jaraco-8.2.1-3.el9.noarch.rp 2.8 MB/s | 11 kB 00:00 2026-02-13T02:18:56.336 INFO:teuthology.orchestra.run.trial107.stdout:(139/173): python3-jaraco-classes-3.2.1-5.el9.n 5.2 MB/s | 18 kB 00:00 2026-02-13T02:18:56.340 INFO:teuthology.orchestra.run.trial107.stdout:(140/173): python3-jaraco-collections-3.0.0-8.e 6.7 MB/s | 23 kB 00:00 2026-02-13T02:18:56.344 INFO:teuthology.orchestra.run.trial107.stdout:(141/173): python3-jaraco-context-6.0.1-3.el9.n 5.7 MB/s | 20 kB 00:00 2026-02-13T02:18:56.347 INFO:teuthology.orchestra.run.trial107.stdout:(142/173): python3-jaraco-functools-3.5.0-2.el9 5.4 MB/s | 19 kB 00:00 2026-02-13T02:18:56.351 INFO:teuthology.orchestra.run.trial107.stdout:(143/173): python3-jaraco-text-4.0.0-2.el9.noar 7.6 MB/s | 26 kB 00:00 2026-02-13T02:18:56.357 INFO:teuthology.orchestra.run.trial107.stdout:(144/173): python3-kubernetes-26.1.0-3.el9.noar 174 MB/s | 1.0 MB 00:00 2026-02-13T02:18:56.361 INFO:teuthology.orchestra.run.trial107.stdout:(145/173): python3-more-itertools-8.12.0-2.el9. 21 MB/s | 79 kB 00:00 2026-02-13T02:18:56.364 INFO:teuthology.orchestra.run.trial107.stdout:(146/173): python3-natsort-7.1.1-5.el9.noarch.r 15 MB/s | 58 kB 00:00 2026-02-13T02:18:56.368 INFO:teuthology.orchestra.run.trial107.stdout:(147/173): python3-portend-3.1.0-2.el9.noarch.r 4.7 MB/s | 16 kB 00:00 2026-02-13T02:18:56.372 INFO:teuthology.orchestra.run.trial107.stdout:(148/173): python3-pyOpenSSL-21.0.0-1.el9.noarc 25 MB/s | 90 kB 00:00 2026-02-13T02:18:56.375 INFO:teuthology.orchestra.run.trial107.stdout:(149/173): python3-pyperclip-1.8.2-2.el9.noarch 8.3 MB/s | 28 kB 00:00 2026-02-13T02:18:56.379 INFO:teuthology.orchestra.run.trial107.stdout:(150/173): python3-repoze-lru-0.7-16.el9.noarch 8.8 MB/s | 31 kB 00:00 2026-02-13T02:18:56.383 INFO:teuthology.orchestra.run.trial107.stdout:(151/173): python3-routes-2.5.1-5.el9.noarch.rp 48 MB/s | 188 kB 00:00 2026-02-13T02:18:56.386 INFO:teuthology.orchestra.run.trial107.stdout:(152/173): python3-rsa-4.9-2.el9.noarch.rpm 17 MB/s | 59 kB 00:00 2026-02-13T02:18:56.390 INFO:teuthology.orchestra.run.trial107.stdout:(153/173): python3-tempora-5.0.0-2.el9.noarch.r 10 MB/s | 36 kB 00:00 2026-02-13T02:18:56.393 INFO:teuthology.orchestra.run.trial107.stdout:(154/173): python3-typing-extensions-4.15.0-1.e 24 MB/s | 86 kB 00:00 2026-02-13T02:18:56.397 INFO:teuthology.orchestra.run.trial107.stdout:(155/173): python3-websocket-client-1.2.3-2.el9 24 MB/s | 90 kB 00:00 2026-02-13T02:18:56.401 INFO:teuthology.orchestra.run.trial107.stdout:(156/173): python3-xmltodict-0.12.0-15.el9.noar 6.4 MB/s | 22 kB 00:00 2026-02-13T02:18:56.404 INFO:teuthology.orchestra.run.trial107.stdout:(157/173): python3-zc-lockfile-2.0-10.el9.noarc 5.8 MB/s | 20 kB 00:00 2026-02-13T02:18:56.408 INFO:teuthology.orchestra.run.trial107.stdout:(158/173): re2-20211101-20.el9.x86_64.rpm 51 MB/s | 191 kB 00:00 2026-02-13T02:18:56.415 INFO:teuthology.orchestra.run.trial107.stdout:(159/173): thrift-0.15.0-4.el9.x86_64.rpm 227 MB/s | 1.6 MB 00:00 2026-02-13T02:18:56.416 INFO:teuthology.orchestra.run.trial027.stdout:(86/173): postgresql-contrib-13.23-1.el9.x86_64 880 kB/s | 903 kB 00:01 2026-02-13T02:18:56.419 INFO:teuthology.orchestra.run.trial107.stdout:(160/173): yaml-cpp-0.6.3-5.el9.x86_64.rpm 32 MB/s | 122 kB 00:00 2026-02-13T02:18:56.564 INFO:teuthology.orchestra.run.trial107.stdout:(161/173): lua-devel-5.4.4-4.el9.x86_64.rpm 40 kB/s | 22 kB 00:00 2026-02-13T02:18:56.574 INFO:teuthology.orchestra.run.trial107.stdout:(162/173): librados2-20.3.0-5241.g8f8d3829.el9. 22 MB/s | 3.5 MB 00:00 2026-02-13T02:18:56.578 INFO:teuthology.orchestra.run.trial027.stdout:(87/173): postgresql-private-libs-13.23-1.el9.x 851 kB/s | 138 kB 00:00 2026-02-13T02:18:56.762 INFO:teuthology.orchestra.run.trial107.stdout:(163/173): librbd1-20.3.0-5241.g8f8d3829.el9.x8 14 MB/s | 2.8 MB 00:00 2026-02-13T02:18:56.935 INFO:teuthology.orchestra.run.trial107.stdout:(164/173): protobuf-compiler-3.14.0-17.el9.x86_ 952 kB/s | 862 kB 00:00 2026-02-13T02:18:57.111 INFO:teuthology.orchestra.run.trial107.stdout:(165/173): NetworkManager-team-1.54.3-2.el9.x86 210 kB/s | 37 kB 00:00 2026-02-13T02:18:57.412 INFO:teuthology.orchestra.run.trial027.stdout:(88/173): openssl-devel-3.5.5-1.el9.x86_64.rpm 900 kB/s | 4.8 MB 00:05 2026-02-13T02:18:57.440 INFO:teuthology.orchestra.run.trial028.stdout:CentOS Stream 9 - CRB 27 MB/s | 7.8 MB 00:00 2026-02-13T02:18:57.468 INFO:teuthology.orchestra.run.trial107.stdout:(166/173): NetworkManager-tui-1.54.3-2.el9.x86_ 695 kB/s | 248 kB 00:00 2026-02-13T02:18:57.900 INFO:teuthology.orchestra.run.trial027.stdout:(89/173): openblas-openmp-0.3.29-1.el9.x86_64.r 902 kB/s | 5.3 MB 00:05 2026-02-13T02:18:58.517 INFO:teuthology.orchestra.run.trial107.stdout:(167/173): iproute-6.17.0-2.el9.x86_64.rpm 826 kB/s | 866 kB 00:01 2026-02-13T02:18:58.585 INFO:teuthology.orchestra.run.trial027.stdout:(90/173): protobuf-3.14.0-17.el9.x86_64.rpm 877 kB/s | 1.0 MB 00:01 2026-02-13T02:18:58.668 INFO:teuthology.orchestra.run.trial027.stdout:(91/173): python3-colorama-0.4.6-3.el9.noarch.r 737 kB/s | 61 kB 00:00 2026-02-13T02:18:58.985 INFO:teuthology.orchestra.run.trial027.stdout:(92/173): python3-devel-3.9.25-2.el9.x86_64.rpm 773 kB/s | 244 kB 00:00 2026-02-13T02:18:59.026 INFO:teuthology.orchestra.run.trial028.stdout:CentOS Stream 9 - Extras packages 23 kB/s | 20 kB 00:00 2026-02-13T02:18:59.106 INFO:teuthology.orchestra.run.trial107.stdout:(168/173): iproute-tc-6.17.0-2.el9.x86_64.rpm 771 kB/s | 454 kB 00:00 2026-02-13T02:18:59.188 INFO:teuthology.orchestra.run.trial107.stdout:(169/173): NetworkManager-libnm-1.54.3-2.el9.x8 804 kB/s | 1.9 MB 00:02 2026-02-13T02:18:59.302 INFO:teuthology.orchestra.run.trial027.stdout:(93/173): python3-jinja2-2.11.3-8.el9.noarch.rp 785 kB/s | 249 kB 00:00 2026-02-13T02:18:59.385 INFO:teuthology.orchestra.run.trial027.stdout:(94/173): python3-jmespath-1.0.1-1.el9.noarch.r 571 kB/s | 48 kB 00:00 2026-02-13T02:18:59.502 INFO:teuthology.orchestra.run.trial107.stdout:(170/173): NetworkManager-1.54.3-2.el9.x86_64.r 831 kB/s | 2.4 MB 00:02 2026-02-13T02:18:59.624 INFO:teuthology.orchestra.run.trial027.stdout:(95/173): python3-libstoragemgmt-1.10.1-1.el9.x 740 kB/s | 177 kB 00:00 2026-02-13T02:18:59.707 INFO:teuthology.orchestra.run.trial027.stdout:(96/173): python3-markupsafe-1.1.1-12.el9.x86_6 420 kB/s | 35 kB 00:00 2026-02-13T02:19:00.181 INFO:teuthology.orchestra.run.trial107.stdout:(171/173): openssl-fips-provider-3.5.5-1.el9.x8 819 kB/s | 814 kB 00:00 2026-02-13T02:19:00.306 INFO:teuthology.orchestra.run.trial028.stdout:Extra Packages for Enterprise Linux 17 MB/s | 20 MB 00:01 2026-02-13T02:19:00.932 INFO:teuthology.orchestra.run.trial107.stdout:(172/173): openssl-3.5.5-1.el9.x86_64.rpm 833 kB/s | 1.5 MB 00:01 2026-02-13T02:19:02.194 INFO:teuthology.orchestra.run.trial107.stdout:(173/173): openssl-libs-3.5.5-1.el9.x86_64.rpm 877 kB/s | 2.3 MB 00:02 2026-02-13T02:19:02.200 INFO:teuthology.orchestra.run.trial107.stdout:-------------------------------------------------------------------------------- 2026-02-13T02:19:02.200 INFO:teuthology.orchestra.run.trial107.stdout:Total 17 MB/s | 302 MB 00:18 2026-02-13T02:19:02.990 INFO:teuthology.orchestra.run.trial107.stdout:Running transaction check 2026-02-13T02:19:03.061 INFO:teuthology.orchestra.run.trial107.stdout:Transaction check succeeded. 2026-02-13T02:19:03.061 INFO:teuthology.orchestra.run.trial107.stdout:Running transaction test 2026-02-13T02:19:03.260 INFO:teuthology.orchestra.run.trial027.stdout:(97/173): postgresql-server-13.23-1.el9.x86_64. 885 kB/s | 5.8 MB 00:06 2026-02-13T02:19:03.488 INFO:teuthology.orchestra.run.trial028.stdout:lab-extras 2.3 MB/s | 50 kB 00:00 2026-02-13T02:19:03.815 INFO:teuthology.orchestra.run.trial027.stdout:(98/173): python3-numpy-f2py-1.23.5-2.el9.x86_6 797 kB/s | 442 kB 00:00 2026-02-13T02:19:03.977 INFO:teuthology.orchestra.run.trial027.stdout:(99/173): python3-packaging-20.9-5.el9.noarch.r 476 kB/s | 77 kB 00:00 2026-02-13T02:19:04.146 INFO:teuthology.orchestra.run.trial107.stdout:Transaction test succeeded. 2026-02-13T02:19:04.146 INFO:teuthology.orchestra.run.trial107.stdout:Running transaction 2026-02-13T02:19:04.297 INFO:teuthology.orchestra.run.trial027.stdout:(100/173): python3-protobuf-3.14.0-17.el9.noarc 837 kB/s | 267 kB 00:00 2026-02-13T02:19:04.526 INFO:teuthology.orchestra.run.trial027.stdout:(101/173): python3-babel-2.9.1-2.el9.noarch.rpm 921 kB/s | 6.0 MB 00:06 2026-02-13T02:19:04.537 INFO:teuthology.orchestra.run.trial027.stdout:(102/173): python3-pyasn1-0.4.8-6.el9.noarch.rp 659 kB/s | 159 kB 00:00 2026-02-13T02:19:04.610 INFO:teuthology.orchestra.run.trial027.stdout:(103/173): python3-pyasn1-modules-0.4.8-6.el9.n 3.2 MB/s | 279 kB 00:00 2026-02-13T02:19:04.621 INFO:teuthology.orchestra.run.trial027.stdout:(104/173): python3-requests-oauthlib-1.3.0-12.e 645 kB/s | 54 kB 00:00 2026-02-13T02:19:04.704 INFO:teuthology.orchestra.run.trial027.stdout:(105/173): python3-toml-0.10.2-6.el9.noarch.rpm 499 kB/s | 42 kB 00:00 2026-02-13T02:19:04.765 INFO:teuthology.orchestra.run.trial028.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2026-02-13T02:19:04.765 INFO:teuthology.orchestra.run.trial028.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2026-02-13T02:19:04.771 INFO:teuthology.orchestra.run.trial028.stdout:Package NetworkManager-1:1.54.1-1.el9.x86_64 is already installed. 2026-02-13T02:19:04.771 INFO:teuthology.orchestra.run.trial028.stdout:Package iproute-6.17.0-1.el9.x86_64 is already installed. 2026-02-13T02:19:04.772 INFO:teuthology.orchestra.run.trial028.stdout:Package util-linux-2.37.4-21.el9.x86_64 is already installed. 2026-02-13T02:19:04.772 INFO:teuthology.orchestra.run.trial028.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2026-02-13T02:19:04.773 INFO:teuthology.orchestra.run.trial028.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2026-02-13T02:19:04.774 INFO:teuthology.orchestra.run.trial028.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2026-02-13T02:19:04.775 INFO:teuthology.orchestra.run.trial028.stdout:Package xfsprogs-6.4.0-7.el9.x86_64 is already installed. 2026-02-13T02:19:04.788 INFO:teuthology.orchestra.run.trial027.stdout:(106/173): python3-wcwidth-0.2.5-8.el9.noarch.r 529 kB/s | 44 kB 00:00 2026-02-13T02:19:04.802 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-13T02:19:04.806 INFO:teuthology.orchestra.run.trial028.stdout:======================================================================================= 2026-02-13T02:19:04.807 INFO:teuthology.orchestra.run.trial028.stdout: Package Arch Version Repository Size 2026-02-13T02:19:04.807 INFO:teuthology.orchestra.run.trial028.stdout:======================================================================================= 2026-02-13T02:19:04.807 INFO:teuthology.orchestra.run.trial028.stdout:Installing: 2026-02-13T02:19:04.807 INFO:teuthology.orchestra.run.trial028.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2026-02-13T02:19:04.807 INFO:teuthology.orchestra.run.trial028.stdout: ceph x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 6.5 k 2026-02-13T02:19:04.807 INFO:teuthology.orchestra.run.trial028.stdout: ceph-base x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 5.2 M 2026-02-13T02:19:04.807 INFO:teuthology.orchestra.run.trial028.stdout: ceph-fuse x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 987 k 2026-02-13T02:19:04.807 INFO:teuthology.orchestra.run.trial028.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 147 k 2026-02-13T02:19:04.807 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 940 k 2026-02-13T02:19:04.807 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5241.g8f8d3829.el9 ceph-noarch 183 k 2026-02-13T02:19:04.807 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5241.g8f8d3829.el9 ceph-noarch 15 M 2026-02-13T02:19:04.808 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5241.g8f8d3829.el9 ceph-noarch 7.4 M 2026-02-13T02:19:04.808 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-rook noarch 2:20.3.0-5241.g8f8d3829.el9 ceph-noarch 50 k 2026-02-13T02:19:04.808 INFO:teuthology.orchestra.run.trial028.stdout: ceph-osd x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 166 k 2026-02-13T02:19:04.808 INFO:teuthology.orchestra.run.trial028.stdout: ceph-osd-classic x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 16 M 2026-02-13T02:19:04.808 INFO:teuthology.orchestra.run.trial028.stdout: ceph-radosgw x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 13 M 2026-02-13T02:19:04.808 INFO:teuthology.orchestra.run.trial028.stdout: ceph-test x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 78 M 2026-02-13T02:19:04.808 INFO:teuthology.orchestra.run.trial028.stdout: ceph-volume noarch 2:20.3.0-5241.g8f8d3829.el9 ceph-noarch 299 k 2026-02-13T02:19:04.808 INFO:teuthology.orchestra.run.trial028.stdout: cephadm noarch 2:20.3.0-5241.g8f8d3829.el9 ceph-noarch 1.0 M 2026-02-13T02:19:04.808 INFO:teuthology.orchestra.run.trial028.stdout: cephfs-mirror x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 243 k 2026-02-13T02:19:04.808 INFO:teuthology.orchestra.run.trial028.stdout: cephfs-shell x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 27 k 2026-02-13T02:19:04.808 INFO:teuthology.orchestra.run.trial028.stdout: cephfs-top noarch 2:20.3.0-5241.g8f8d3829.el9 ceph-noarch 18 k 2026-02-13T02:19:04.808 INFO:teuthology.orchestra.run.trial028.stdout: elfutils-libelf-devel x86_64 0.194-1.el9 appstream 79 k 2026-02-13T02:19:04.809 INFO:teuthology.orchestra.run.trial028.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2026-02-13T02:19:04.809 INFO:teuthology.orchestra.run.trial028.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2026-02-13T02:19:04.809 INFO:teuthology.orchestra.run.trial028.stdout: libcephfs-devel x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 37 k 2026-02-13T02:19:04.809 INFO:teuthology.orchestra.run.trial028.stdout: libcephfs2 x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 909 k 2026-02-13T02:19:04.809 INFO:teuthology.orchestra.run.trial028.stdout: librados-devel x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 123 k 2026-02-13T02:19:04.809 INFO:teuthology.orchestra.run.trial028.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2026-02-13T02:19:04.809 INFO:teuthology.orchestra.run.trial028.stdout: libuuid-devel x86_64 2.37.4-21.el9 appstream 25 k 2026-02-13T02:19:04.809 INFO:teuthology.orchestra.run.trial028.stdout: openssl-devel x86_64 1:3.5.5-1.el9 appstream 4.8 M 2026-02-13T02:19:04.809 INFO:teuthology.orchestra.run.trial028.stdout: postgresql x86_64 13.23-1.el9 appstream 1.6 M 2026-02-13T02:19:04.809 INFO:teuthology.orchestra.run.trial028.stdout: postgresql-contrib x86_64 13.23-1.el9 appstream 903 k 2026-02-13T02:19:04.809 INFO:teuthology.orchestra.run.trial028.stdout: postgresql-server x86_64 13.23-1.el9 appstream 5.8 M 2026-02-13T02:19:04.809 INFO:teuthology.orchestra.run.trial028.stdout: python3-cephfs x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 204 k 2026-02-13T02:19:04.809 INFO:teuthology.orchestra.run.trial028.stdout: python3-rados x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 324 k 2026-02-13T02:19:04.810 INFO:teuthology.orchestra.run.trial028.stdout: python3-rbd x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 306 k 2026-02-13T02:19:04.810 INFO:teuthology.orchestra.run.trial028.stdout: python3-rgw x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 100 k 2026-02-13T02:19:04.810 INFO:teuthology.orchestra.run.trial028.stdout: rbd-fuse x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 84 k 2026-02-13T02:19:04.810 INFO:teuthology.orchestra.run.trial028.stdout: rbd-mirror x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 2.9 M 2026-02-13T02:19:04.810 INFO:teuthology.orchestra.run.trial028.stdout: rbd-nbd x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 172 k 2026-02-13T02:19:04.810 INFO:teuthology.orchestra.run.trial028.stdout: xfsprogs-devel x86_64 6.4.0-7.el9 appstream 96 k 2026-02-13T02:19:04.810 INFO:teuthology.orchestra.run.trial028.stdout:Upgrading: 2026-02-13T02:19:04.810 INFO:teuthology.orchestra.run.trial028.stdout: NetworkManager x86_64 1:1.54.3-2.el9 baseos 2.4 M 2026-02-13T02:19:04.810 INFO:teuthology.orchestra.run.trial028.stdout: NetworkManager-libnm x86_64 1:1.54.3-2.el9 baseos 1.9 M 2026-02-13T02:19:04.810 INFO:teuthology.orchestra.run.trial028.stdout: NetworkManager-team x86_64 1:1.54.3-2.el9 baseos 37 k 2026-02-13T02:19:04.810 INFO:teuthology.orchestra.run.trial028.stdout: NetworkManager-tui x86_64 1:1.54.3-2.el9 baseos 248 k 2026-02-13T02:19:04.810 INFO:teuthology.orchestra.run.trial028.stdout: iproute x86_64 6.17.0-2.el9 baseos 866 k 2026-02-13T02:19:04.811 INFO:teuthology.orchestra.run.trial028.stdout: iproute-tc x86_64 6.17.0-2.el9 baseos 454 k 2026-02-13T02:19:04.811 INFO:teuthology.orchestra.run.trial028.stdout: librados2 x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 3.5 M 2026-02-13T02:19:04.811 INFO:teuthology.orchestra.run.trial028.stdout: librbd1 x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 2.8 M 2026-02-13T02:19:04.811 INFO:teuthology.orchestra.run.trial028.stdout: openssl x86_64 1:3.5.5-1.el9 baseos 1.5 M 2026-02-13T02:19:04.811 INFO:teuthology.orchestra.run.trial028.stdout: openssl-fips-provider x86_64 1:3.5.5-1.el9 baseos 814 k 2026-02-13T02:19:04.811 INFO:teuthology.orchestra.run.trial028.stdout: openssl-libs x86_64 1:3.5.5-1.el9 baseos 2.3 M 2026-02-13T02:19:04.811 INFO:teuthology.orchestra.run.trial028.stdout:Installing dependencies: 2026-02-13T02:19:04.811 INFO:teuthology.orchestra.run.trial028.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-13T02:19:04.811 INFO:teuthology.orchestra.run.trial028.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2026-02-13T02:19:04.811 INFO:teuthology.orchestra.run.trial028.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-13T02:19:04.811 INFO:teuthology.orchestra.run.trial028.stdout: ceph-common x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 24 M 2026-02-13T02:19:04.811 INFO:teuthology.orchestra.run.trial028.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5241.g8f8d3829.el9 ceph-noarch 57 k 2026-02-13T02:19:04.811 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mds x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 2.4 M 2026-02-13T02:19:04.812 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5241.g8f8d3829.el9 ceph-noarch 297 k 2026-02-13T02:19:04.812 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mon x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 5.0 M 2026-02-13T02:19:04.812 INFO:teuthology.orchestra.run.trial028.stdout: ceph-osd-crimson x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 19 M 2026-02-13T02:19:04.812 INFO:teuthology.orchestra.run.trial028.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5241.g8f8d3829.el9 ceph-noarch 17 k 2026-02-13T02:19:04.812 INFO:teuthology.orchestra.run.trial028.stdout: ceph-selinux x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 25 k 2026-02-13T02:19:04.812 INFO:teuthology.orchestra.run.trial028.stdout: cryptsetup x86_64 2.8.1-2.el9 baseos 350 k 2026-02-13T02:19:04.812 INFO:teuthology.orchestra.run.trial028.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2026-02-13T02:19:04.812 INFO:teuthology.orchestra.run.trial028.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2026-02-13T02:19:04.812 INFO:teuthology.orchestra.run.trial028.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2026-02-13T02:19:04.812 INFO:teuthology.orchestra.run.trial028.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-13T02:19:04.812 INFO:teuthology.orchestra.run.trial028.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-13T02:19:04.812 INFO:teuthology.orchestra.run.trial028.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-13T02:19:04.813 INFO:teuthology.orchestra.run.trial028.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-13T02:19:04.813 INFO:teuthology.orchestra.run.trial028.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-13T02:19:04.813 INFO:teuthology.orchestra.run.trial028.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-13T02:19:04.813 INFO:teuthology.orchestra.run.trial028.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-13T02:19:04.813 INFO:teuthology.orchestra.run.trial028.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 24 k 2026-02-13T02:19:04.813 INFO:teuthology.orchestra.run.trial028.stdout: libcephsqlite x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 159 k 2026-02-13T02:19:04.813 INFO:teuthology.orchestra.run.trial028.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-13T02:19:04.813 INFO:teuthology.orchestra.run.trial028.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-13T02:19:04.813 INFO:teuthology.orchestra.run.trial028.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-13T02:19:04.813 INFO:teuthology.orchestra.run.trial028.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-13T02:19:04.813 INFO:teuthology.orchestra.run.trial028.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-13T02:19:04.813 INFO:teuthology.orchestra.run.trial028.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-13T02:19:04.813 INFO:teuthology.orchestra.run.trial028.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-13T02:19:04.814 INFO:teuthology.orchestra.run.trial028.stdout: libradosstriper1 x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 248 k 2026-02-13T02:19:04.814 INFO:teuthology.orchestra.run.trial028.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-13T02:19:04.814 INFO:teuthology.orchestra.run.trial028.stdout: librgw2 x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 6.5 M 2026-02-13T02:19:04.814 INFO:teuthology.orchestra.run.trial028.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-13T02:19:04.814 INFO:teuthology.orchestra.run.trial028.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-13T02:19:04.814 INFO:teuthology.orchestra.run.trial028.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-13T02:19:04.814 INFO:teuthology.orchestra.run.trial028.stdout: libzstd-devel x86_64 1.5.5-1.el9 appstream 51 k 2026-02-13T02:19:04.814 INFO:teuthology.orchestra.run.trial028.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-13T02:19:04.814 INFO:teuthology.orchestra.run.trial028.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-13T02:19:04.814 INFO:teuthology.orchestra.run.trial028.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-13T02:19:04.814 INFO:teuthology.orchestra.run.trial028.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-13T02:19:04.814 INFO:teuthology.orchestra.run.trial028.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-13T02:19:04.815 INFO:teuthology.orchestra.run.trial028.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-13T02:19:04.815 INFO:teuthology.orchestra.run.trial028.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-13T02:19:04.815 INFO:teuthology.orchestra.run.trial028.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-13T02:19:04.815 INFO:teuthology.orchestra.run.trial028.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-13T02:19:04.815 INFO:teuthology.orchestra.run.trial028.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-13T02:19:04.815 INFO:teuthology.orchestra.run.trial028.stdout: perl-Thread-Queue noarch 3.14-460.el9 appstream 22 k 2026-02-13T02:19:04.815 INFO:teuthology.orchestra.run.trial028.stdout: perl-threads x86_64 1:2.25-460.el9 appstream 58 k 2026-02-13T02:19:04.815 INFO:teuthology.orchestra.run.trial028.stdout: perl-threads-shared x86_64 1.61-460.el9 appstream 45 k 2026-02-13T02:19:04.815 INFO:teuthology.orchestra.run.trial028.stdout: postgresql-private-libs x86_64 13.23-1.el9 appstream 138 k 2026-02-13T02:19:04.815 INFO:teuthology.orchestra.run.trial028.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-13T02:19:04.815 INFO:teuthology.orchestra.run.trial028.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-13T02:19:04.815 INFO:teuthology.orchestra.run.trial028.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-13T02:19:04.815 INFO:teuthology.orchestra.run.trial028.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-13T02:19:04.816 INFO:teuthology.orchestra.run.trial028.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-13T02:19:04.816 INFO:teuthology.orchestra.run.trial028.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-13T02:19:04.816 INFO:teuthology.orchestra.run.trial028.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-13T02:19:04.816 INFO:teuthology.orchestra.run.trial028.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-13T02:19:04.816 INFO:teuthology.orchestra.run.trial028.stdout: python3-ceph-argparse x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 45 k 2026-02-13T02:19:04.816 INFO:teuthology.orchestra.run.trial028.stdout: python3-ceph-common x86_64 2:20.3.0-5241.g8f8d3829.el9 ceph 185 k 2026-02-13T02:19:04.816 INFO:teuthology.orchestra.run.trial028.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-13T02:19:04.816 INFO:teuthology.orchestra.run.trial028.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-13T02:19:04.816 INFO:teuthology.orchestra.run.trial028.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-13T02:19:04.816 INFO:teuthology.orchestra.run.trial028.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-13T02:19:04.816 INFO:teuthology.orchestra.run.trial028.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2026-02-13T02:19:04.816 INFO:teuthology.orchestra.run.trial028.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2026-02-13T02:19:04.817 INFO:teuthology.orchestra.run.trial028.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-13T02:19:04.817 INFO:teuthology.orchestra.run.trial028.stdout: python3-devel x86_64 3.9.25-2.el9 appstream 244 k 2026-02-13T02:19:04.817 INFO:teuthology.orchestra.run.trial028.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-13T02:19:04.817 INFO:teuthology.orchestra.run.trial028.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-13T02:19:04.817 INFO:teuthology.orchestra.run.trial028.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-13T02:19:04.817 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-13T02:19:04.817 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-13T02:19:04.817 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-13T02:19:04.817 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-13T02:19:04.817 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-13T02:19:04.817 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-13T02:19:04.817 INFO:teuthology.orchestra.run.trial028.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-13T02:19:04.818 INFO:teuthology.orchestra.run.trial028.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-13T02:19:04.818 INFO:teuthology.orchestra.run.trial028.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-13T02:19:04.818 INFO:teuthology.orchestra.run.trial028.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-13T02:19:04.818 INFO:teuthology.orchestra.run.trial028.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-13T02:19:04.818 INFO:teuthology.orchestra.run.trial028.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-13T02:19:04.818 INFO:teuthology.orchestra.run.trial028.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-13T02:19:04.818 INFO:teuthology.orchestra.run.trial028.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-13T02:19:04.818 INFO:teuthology.orchestra.run.trial028.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-13T02:19:04.818 INFO:teuthology.orchestra.run.trial028.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-13T02:19:04.818 INFO:teuthology.orchestra.run.trial028.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-13T02:19:04.818 INFO:teuthology.orchestra.run.trial028.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-13T02:19:04.818 INFO:teuthology.orchestra.run.trial028.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-13T02:19:04.819 INFO:teuthology.orchestra.run.trial028.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-13T02:19:04.819 INFO:teuthology.orchestra.run.trial028.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-13T02:19:04.819 INFO:teuthology.orchestra.run.trial028.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-13T02:19:04.819 INFO:teuthology.orchestra.run.trial028.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-13T02:19:04.819 INFO:teuthology.orchestra.run.trial028.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2026-02-13T02:19:04.819 INFO:teuthology.orchestra.run.trial028.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-13T02:19:04.819 INFO:teuthology.orchestra.run.trial028.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-13T02:19:04.819 INFO:teuthology.orchestra.run.trial028.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-13T02:19:04.819 INFO:teuthology.orchestra.run.trial028.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-13T02:19:04.819 INFO:teuthology.orchestra.run.trial028.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-13T02:19:04.819 INFO:teuthology.orchestra.run.trial028.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-13T02:19:04.819 INFO:teuthology.orchestra.run.trial028.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-13T02:19:04.819 INFO:teuthology.orchestra.run.trial028.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-13T02:19:04.820 INFO:teuthology.orchestra.run.trial028.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-13T02:19:04.820 INFO:teuthology.orchestra.run.trial028.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-13T02:19:04.820 INFO:teuthology.orchestra.run.trial028.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2026-02-13T02:19:04.820 INFO:teuthology.orchestra.run.trial028.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-13T02:19:04.820 INFO:teuthology.orchestra.run.trial028.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-13T02:19:04.820 INFO:teuthology.orchestra.run.trial028.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-13T02:19:04.820 INFO:teuthology.orchestra.run.trial028.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-13T02:19:04.820 INFO:teuthology.orchestra.run.trial028.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-13T02:19:04.820 INFO:teuthology.orchestra.run.trial028.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-13T02:19:04.820 INFO:teuthology.orchestra.run.trial028.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-13T02:19:04.820 INFO:teuthology.orchestra.run.trial028.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-13T02:19:04.820 INFO:teuthology.orchestra.run.trial028.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-13T02:19:04.820 INFO:teuthology.orchestra.run.trial028.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2026-02-13T02:19:04.820 INFO:teuthology.orchestra.run.trial028.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-13T02:19:04.821 INFO:teuthology.orchestra.run.trial028.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-13T02:19:04.821 INFO:teuthology.orchestra.run.trial028.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-13T02:19:04.821 INFO:teuthology.orchestra.run.trial028.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2026-02-13T02:19:04.821 INFO:teuthology.orchestra.run.trial028.stdout:Installing weak dependencies: 2026-02-13T02:19:04.821 INFO:teuthology.orchestra.run.trial028.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-13T02:19:04.821 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:19:04.821 INFO:teuthology.orchestra.run.trial028.stdout:Transaction Summary 2026-02-13T02:19:04.821 INFO:teuthology.orchestra.run.trial028.stdout:======================================================================================= 2026-02-13T02:19:04.821 INFO:teuthology.orchestra.run.trial028.stdout:Install 162 Packages 2026-02-13T02:19:04.821 INFO:teuthology.orchestra.run.trial028.stdout:Upgrade 11 Packages 2026-02-13T02:19:04.821 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:19:04.822 INFO:teuthology.orchestra.run.trial028.stdout:Total download size: 302 M 2026-02-13T02:19:04.822 INFO:teuthology.orchestra.run.trial028.stdout:Downloading Packages: 2026-02-13T02:19:05.106 INFO:teuthology.orchestra.run.trial027.stdout:(107/173): qatlib-25.08.0-2.el9.x86_64.rpm 752 kB/s | 240 kB 00:00 2026-02-13T02:19:05.162 INFO:teuthology.orchestra.run.trial028.stdout:(1/173): ceph-20.3.0-5241.g8f8d3829.el9.x86_64. 97 kB/s | 6.5 kB 00:00 2026-02-13T02:19:05.189 INFO:teuthology.orchestra.run.trial027.stdout:(108/173): qatlib-service-25.08.0-2.el9.x86_64. 447 kB/s | 37 kB 00:00 2026-02-13T02:19:05.272 INFO:teuthology.orchestra.run.trial027.stdout:(109/173): qatzip-libs-1.3.1-1.el9.x86_64.rpm 799 kB/s | 66 kB 00:00 2026-02-13T02:19:05.282 INFO:teuthology.orchestra.run.trial028.stdout:(2/173): ceph-fuse-20.3.0-5241.g8f8d3829.el9.x8 8.1 MB/s | 987 kB 00:00 2026-02-13T02:19:05.302 INFO:teuthology.orchestra.run.trial107.stdout: Preparing : 1/1 2026-02-13T02:19:05.305 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/184 2026-02-13T02:19:05.313 INFO:teuthology.orchestra.run.trial107.stdout: Installing : liboath-2.6.12-1.el9.x86_64 2/184 2026-02-13T02:19:05.324 INFO:teuthology.orchestra.run.trial107.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/184 2026-02-13T02:19:05.328 INFO:teuthology.orchestra.run.trial107.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/184 2026-02-13T02:19:05.329 INFO:teuthology.orchestra.run.trial107.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 5/184 2026-02-13T02:19:05.330 INFO:teuthology.orchestra.run.trial028.stdout:(3/173): ceph-immutable-object-cache-20.3.0-524 3.0 MB/s | 147 kB 00:00 2026-02-13T02:19:05.334 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/184 2026-02-13T02:19:05.374 INFO:teuthology.orchestra.run.trial107.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 7/184 2026-02-13T02:19:05.378 INFO:teuthology.orchestra.run.trial107.stdout: Upgrading : NetworkManager-libnm-1:1.54.3-2.el9.x86_64 8/184 2026-02-13T02:19:05.411 INFO:teuthology.orchestra.run.trial028.stdout:(4/173): ceph-base-20.3.0-5241.g8f8d3829.el9.x8 16 MB/s | 5.2 MB 00:00 2026-02-13T02:19:05.412 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: NetworkManager-1:1.54.3-2.el9.x86_64 9/184 2026-02-13T02:19:05.415 INFO:teuthology.orchestra.run.trial107.stdout: Upgrading : NetworkManager-1:1.54.3-2.el9.x86_64 9/184 2026-02-13T02:19:05.470 INFO:teuthology.orchestra.run.trial028.stdout:(5/173): ceph-mgr-20.3.0-5241.g8f8d3829.el9.x86 16 MB/s | 940 kB 00:00 2026-02-13T02:19:05.517 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: NetworkManager-1:1.54.3-2.el9.x86_64 9/184 2026-02-13T02:19:05.520 INFO:teuthology.orchestra.run.trial107.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 10/184 2026-02-13T02:19:05.550 INFO:teuthology.orchestra.run.trial107.stdout: Installing : re2-1:20211101-20.el9.x86_64 11/184 2026-02-13T02:19:05.557 INFO:teuthology.orchestra.run.trial027.stdout:(110/173): python3-numpy-1.23.5-2.el9.x86_64.rp 1.0 MB/s | 6.1 MB 00:05 2026-02-13T02:19:05.565 INFO:teuthology.orchestra.run.trial028.stdout:(6/173): ceph-mds-20.3.0-5241.g8f8d3829.el9.x86 10 MB/s | 2.4 MB 00:00 2026-02-13T02:19:05.572 INFO:teuthology.orchestra.run.trial107.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 12/184 2026-02-13T02:19:05.583 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 13/184 2026-02-13T02:19:05.586 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-packaging-20.9-5.el9.noarch 14/184 2026-02-13T02:19:05.592 INFO:teuthology.orchestra.run.trial107.stdout: Installing : perl-threads-1:2.25-460.el9.x86_64 15/184 2026-02-13T02:19:05.597 INFO:teuthology.orchestra.run.trial107.stdout: Installing : lua-5.4.4-4.el9.x86_64 16/184 2026-02-13T02:19:05.600 INFO:teuthology.orchestra.run.trial107.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 17/184 2026-02-13T02:19:05.604 INFO:teuthology.orchestra.run.trial107.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 18/184 2026-02-13T02:19:05.625 INFO:teuthology.orchestra.run.trial028.stdout:(7/173): ceph-osd-20.3.0-5241.g8f8d3829.el9.x86 2.7 MB/s | 166 kB 00:00 2026-02-13T02:19:05.627 INFO:teuthology.orchestra.run.trial107.stdout: Installing : unzip-6.0-59.el9.x86_64 19/184 2026-02-13T02:19:05.640 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 20/184 2026-02-13T02:19:05.646 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 21/184 2026-02-13T02:19:05.649 INFO:teuthology.orchestra.run.trial107.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 22/184 2026-02-13T02:19:05.655 INFO:teuthology.orchestra.run.trial107.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 23/184 2026-02-13T02:19:05.657 INFO:teuthology.orchestra.run.trial107.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 24/184 2026-02-13T02:19:05.659 INFO:teuthology.orchestra.run.trial107.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 25/184 2026-02-13T02:19:05.668 INFO:teuthology.orchestra.run.trial107.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 26/184 2026-02-13T02:19:05.670 INFO:teuthology.orchestra.run.trial027.stdout:(111/173): socat-1.7.4.1-8.el9.x86_64.rpm 764 kB/s | 303 kB 00:00 2026-02-13T02:19:05.703 INFO:teuthology.orchestra.run.trial107.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 27/184 2026-02-13T02:19:05.708 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-ceph-common-2:20.3.0-5241.g8f8d3829.el9. 28/184 2026-02-13T02:19:05.710 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-ceph-argparse-2:20.3.0-5241.g8f8d3829.el 29/184 2026-02-13T02:19:05.710 INFO:teuthology.orchestra.run.trial107.stdout: Installing : libcephfs-proxy2-2:20.3.0-5241.g8f8d3829.el9.x86 30/184 2026-02-13T02:19:05.735 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5241.g8f8d3829.el9.x86 30/184 2026-02-13T02:19:05.742 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 31/184 2026-02-13T02:19:05.766 INFO:teuthology.orchestra.run.trial107.stdout: Installing : zip-3.0-35.el9.x86_64 32/184 2026-02-13T02:19:05.771 INFO:teuthology.orchestra.run.trial107.stdout: Installing : luarocks-3.9.2-5.el9.noarch 33/184 2026-02-13T02:19:05.774 INFO:teuthology.orchestra.run.trial107.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 34/184 2026-02-13T02:19:05.777 INFO:teuthology.orchestra.run.trial107.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/184 2026-02-13T02:19:05.779 INFO:teuthology.orchestra.run.trial028.stdout:(8/173): ceph-mon-20.3.0-5241.g8f8d3829.el9.x86 16 MB/s | 5.0 MB 00:00 2026-02-13T02:19:05.779 INFO:teuthology.orchestra.run.trial107.stdout: Installing : perl-threads-shared-1.61-460.el9.x86_64 36/184 2026-02-13T02:19:05.807 INFO:teuthology.orchestra.run.trial107.stdout: Installing : perl-Thread-Queue-3.14-460.el9.noarch 37/184 2026-02-13T02:19:05.825 INFO:teuthology.orchestra.run.trial027.stdout:(112/173): python3-scipy-1.9.3-2.el9.x86_64.rpm 16 MB/s | 19 MB 00:01 2026-02-13T02:19:05.831 INFO:teuthology.orchestra.run.trial027.stdout:(113/173): xfsprogs-devel-6.4.0-7.el9.x86_64.rp 593 kB/s | 96 kB 00:00 2026-02-13T02:19:05.859 INFO:teuthology.orchestra.run.trial107.stdout: Installing : automake-1.16.2-8.el9.noarch 38/184 2026-02-13T02:19:05.872 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/184 2026-02-13T02:19:05.877 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/184 2026-02-13T02:19:05.885 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 41/184 2026-02-13T02:19:05.902 INFO:teuthology.orchestra.run.trial107.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 42/184 2026-02-13T02:19:05.908 INFO:teuthology.orchestra.run.trial027.stdout:(114/173): xmlstarlet-1.6.1-20.el9.x86_64.rpm 772 kB/s | 64 kB 00:00 2026-02-13T02:19:05.908 INFO:teuthology.orchestra.run.trial107.stdout: Upgrading : openssl-libs-1:3.5.5-1.el9.x86_64 43/184 2026-02-13T02:19:05.914 INFO:teuthology.orchestra.run.trial027.stdout:(115/173): zlib-devel-1.2.11-41.el9.x86_64.rpm 542 kB/s | 45 kB 00:00 2026-02-13T02:19:05.966 INFO:teuthology.orchestra.run.trial027.stdout:(116/173): uuid-1.6.2-55.el9.x86_64.rpm 141 kB/s | 58 kB 00:00 2026-02-13T02:19:05.988 INFO:teuthology.orchestra.run.trial027.stdout:(117/173): lua-devel-5.4.4-4.el9.x86_64.rpm 278 kB/s | 22 kB 00:00 2026-02-13T02:19:06.004 INFO:teuthology.orchestra.run.trial028.stdout:(9/173): ceph-common-20.3.0-5241.g8f8d3829.el9. 27 MB/s | 24 MB 00:00 2026-02-13T02:19:06.065 INFO:teuthology.orchestra.run.trial107.stdout: Upgrading : openssl-fips-provider-1:3.5.5-1.el9.x86_64 44/184 2026-02-13T02:19:06.068 INFO:teuthology.orchestra.run.trial107.stdout: Upgrading : librados2-2:20.3.0-5241.g8f8d3829.el9.x86_64 45/184 2026-02-13T02:19:06.114 INFO:teuthology.orchestra.run.trial027.stdout:(118/173): protobuf-compiler-3.14.0-17.el9.x86_ 4.2 MB/s | 862 kB 00:00 2026-02-13T02:19:06.138 INFO:teuthology.orchestra.run.trial027.stdout:(119/173): gperftools-libs-2.9.1-3.el9.x86_64.r 2.0 MB/s | 308 kB 00:00 2026-02-13T02:19:06.154 INFO:teuthology.orchestra.run.trial027.stdout:(120/173): abseil-cpp-20211102.0-4.el9.x86_64.r 2.9 MB/s | 551 kB 00:00 2026-02-13T02:19:06.175 INFO:teuthology.orchestra.run.trial027.stdout:(121/173): libarrow-doc-9.0.0-15.el9.noarch.rpm 1.2 MB/s | 25 kB 00:00 2026-02-13T02:19:06.196 INFO:teuthology.orchestra.run.trial027.stdout:(122/173): liboath-2.6.12-1.el9.x86_64.rpm 2.3 MB/s | 49 kB 00:00 2026-02-13T02:19:06.200 INFO:teuthology.orchestra.run.trial027.stdout:(123/173): grpc-data-1.46.7-10.el9.noarch.rpm 228 kB/s | 19 kB 00:00 2026-02-13T02:19:06.208 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: librados2-2:20.3.0-5241.g8f8d3829.el9.x86_64 45/184 2026-02-13T02:19:06.209 INFO:teuthology.orchestra.run.trial107.stdout: Upgrading : librbd1-2:20.3.0-5241.g8f8d3829.el9.x86_64 46/184 2026-02-13T02:19:06.218 INFO:teuthology.orchestra.run.trial027.stdout:(124/173): libunwind-1.6.2-1.el9.x86_64.rpm 3.0 MB/s | 67 kB 00:00 2026-02-13T02:19:06.249 INFO:teuthology.orchestra.run.trial027.stdout:(125/173): libarrow-9.0.0-15.el9.x86_64.rpm 40 MB/s | 4.4 MB 00:00 2026-02-13T02:19:06.260 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: librbd1-2:20.3.0-5241.g8f8d3829.el9.x86_64 46/184 2026-02-13T02:19:06.261 INFO:teuthology.orchestra.run.trial107.stdout: Installing : libcephfs2-2:20.3.0-5241.g8f8d3829.el9.x86_64 47/184 2026-02-13T02:19:06.263 INFO:teuthology.orchestra.run.trial027.stdout:(126/173): parquet-libs-9.0.0-15.el9.x86_64.rpm 18 MB/s | 838 kB 00:00 2026-02-13T02:19:06.273 INFO:teuthology.orchestra.run.trial027.stdout:(127/173): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2026-02-13T02:19:06.275 INFO:teuthology.orchestra.run.trial027.stdout:(128/173): python3-asyncssh-2.13.2-5.el9.noarch 21 MB/s | 548 kB 00:00 2026-02-13T02:19:06.284 INFO:teuthology.orchestra.run.trial027.stdout:(129/173): python3-autocommand-2.2.2-8.el9.noar 1.4 MB/s | 29 kB 00:00 2026-02-13T02:19:06.290 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: libcephfs2-2:20.3.0-5241.g8f8d3829.el9.x86_64 47/184 2026-02-13T02:19:06.292 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-rados-2:20.3.0-5241.g8f8d3829.el9.x86_64 48/184 2026-02-13T02:19:06.294 INFO:teuthology.orchestra.run.trial027.stdout:(130/173): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.1 MB/s | 43 kB 00:00 2026-02-13T02:19:06.299 INFO:teuthology.orchestra.run.trial027.stdout:(131/173): python3-backports-tarfile-1.2.0-1.el 2.3 MB/s | 60 kB 00:00 2026-02-13T02:19:06.300 INFO:teuthology.orchestra.run.trial107.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 49/184 2026-02-13T02:19:06.304 INFO:teuthology.orchestra.run.trial107.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 50/184 2026-02-13T02:19:06.305 INFO:teuthology.orchestra.run.trial027.stdout:(132/173): python3-cachetools-4.2.4-1.el9.noarc 1.5 MB/s | 32 kB 00:00 2026-02-13T02:19:06.314 INFO:teuthology.orchestra.run.trial027.stdout:(133/173): python3-certifi-2023.05.07-4.el9.noa 725 kB/s | 14 kB 00:00 2026-02-13T02:19:06.329 INFO:teuthology.orchestra.run.trial027.stdout:(134/173): python3-cherrypy-18.6.1-2.el9.noarch 14 MB/s | 358 kB 00:00 2026-02-13T02:19:06.334 INFO:teuthology.orchestra.run.trial027.stdout:(135/173): python3-cheroot-10.0.1-4.el9.noarch. 4.8 MB/s | 173 kB 00:00 2026-02-13T02:19:06.337 INFO:teuthology.orchestra.run.trial027.stdout:(136/173): python3-cmd2-2.4.2-2.el9.noarch.rpm 13 MB/s | 303 kB 00:00 2026-02-13T02:19:06.352 INFO:teuthology.orchestra.run.trial027.stdout:(137/173): python3-google-auth-2.45.0-1.el9.noa 11 MB/s | 254 kB 00:00 2026-02-13T02:19:06.357 INFO:teuthology.orchestra.run.trial027.stdout:(138/173): python3-grpcio-tools-1.46.7-10.el9.x 6.8 MB/s | 144 kB 00:00 2026-02-13T02:19:06.364 INFO:teuthology.orchestra.run.trial107.stdout: Installing : postgresql-private-libs-13.23-1.el9.x86_64 51/184 2026-02-13T02:19:06.373 INFO:teuthology.orchestra.run.trial027.stdout:(139/173): python3-jaraco-8.2.1-3.el9.noarch.rp 527 kB/s | 11 kB 00:00 2026-02-13T02:19:06.377 INFO:teuthology.orchestra.run.trial027.stdout:(140/173): python3-jaraco-classes-3.2.1-5.el9.n 910 kB/s | 18 kB 00:00 2026-02-13T02:19:06.378 INFO:teuthology.orchestra.run.trial107.stdout: Installing : postgresql-13.23-1.el9.x86_64 52/184 2026-02-13T02:19:06.386 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-cephfs-2:20.3.0-5241.g8f8d3829.el9.x86_6 53/184 2026-02-13T02:19:06.387 INFO:teuthology.orchestra.run.trial107.stdout: Installing : libcephsqlite-2:20.3.0-5241.g8f8d3829.el9.x86_64 54/184 2026-02-13T02:19:06.393 INFO:teuthology.orchestra.run.trial027.stdout:(141/173): python3-jaraco-collections-3.0.0-8.e 1.1 MB/s | 23 kB 00:00 2026-02-13T02:19:06.397 INFO:teuthology.orchestra.run.trial027.stdout:(142/173): python3-jaraco-context-6.0.1-3.el9.n 991 kB/s | 20 kB 00:00 2026-02-13T02:19:06.410 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5241.g8f8d3829.el9.x86_64 54/184 2026-02-13T02:19:06.411 INFO:teuthology.orchestra.run.trial107.stdout: Installing : libradosstriper1-2:20.3.0-5241.g8f8d3829.el9.x86 55/184 2026-02-13T02:19:06.414 INFO:teuthology.orchestra.run.trial027.stdout:(143/173): python3-jaraco-functools-3.5.0-2.el9 945 kB/s | 19 kB 00:00 2026-02-13T02:19:06.417 INFO:teuthology.orchestra.run.trial027.stdout:(144/173): python3-jaraco-text-4.0.0-2.el9.noar 1.3 MB/s | 26 kB 00:00 2026-02-13T02:19:06.427 INFO:teuthology.orchestra.run.trial027.stdout:(145/173): python3-grpcio-1.46.7-10.el9.x86_64. 22 MB/s | 2.0 MB 00:00 2026-02-13T02:19:06.438 INFO:teuthology.orchestra.run.trial027.stdout:(146/173): python3-more-itertools-8.12.0-2.el9. 3.7 MB/s | 79 kB 00:00 2026-02-13T02:19:06.439 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5241.g8f8d3829.el9.x86 55/184 2026-02-13T02:19:06.446 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-rbd-2:20.3.0-5241.g8f8d3829.el9.x86_64 56/184 2026-02-13T02:19:06.447 INFO:teuthology.orchestra.run.trial027.stdout:(147/173): python3-kubernetes-26.1.0-3.el9.noar 32 MB/s | 1.0 MB 00:00 2026-02-13T02:19:06.448 INFO:teuthology.orchestra.run.trial027.stdout:(148/173): python3-natsort-7.1.1-5.el9.noarch.r 2.7 MB/s | 58 kB 00:00 2026-02-13T02:19:06.455 INFO:teuthology.orchestra.run.trial107.stdout: Installing : librados-devel-2:20.3.0-5241.g8f8d3829.el9.x86_6 57/184 2026-02-13T02:19:06.457 INFO:teuthology.orchestra.run.trial027.stdout:(149/173): python3-portend-3.1.0-2.el9.noarch.r 847 kB/s | 16 kB 00:00 2026-02-13T02:19:06.467 INFO:teuthology.orchestra.run.trial107.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 58/184 2026-02-13T02:19:06.467 INFO:teuthology.orchestra.run.trial027.stdout:(150/173): python3-pyOpenSSL-21.0.0-1.el9.noarc 4.2 MB/s | 90 kB 00:00 2026-02-13T02:19:06.469 INFO:teuthology.orchestra.run.trial027.stdout:(151/173): python3-pyperclip-1.8.2-2.el9.noarch 1.3 MB/s | 28 kB 00:00 2026-02-13T02:19:06.477 INFO:teuthology.orchestra.run.trial027.stdout:(152/173): python3-repoze-lru-0.7-16.el9.noarch 1.5 MB/s | 31 kB 00:00 2026-02-13T02:19:06.490 INFO:teuthology.orchestra.run.trial027.stdout:(153/173): python3-rsa-4.9-2.el9.noarch.rpm 2.8 MB/s | 59 kB 00:00 2026-02-13T02:19:06.490 INFO:teuthology.orchestra.run.trial027.stdout:(154/173): python3-routes-2.5.1-5.el9.noarch.rp 8.1 MB/s | 188 kB 00:00 2026-02-13T02:19:06.497 INFO:teuthology.orchestra.run.trial027.stdout:(155/173): python3-tempora-5.0.0-2.el9.noarch.r 1.8 MB/s | 36 kB 00:00 2026-02-13T02:19:06.511 INFO:teuthology.orchestra.run.trial027.stdout:(156/173): python3-typing-extensions-4.15.0-1.e 4.0 MB/s | 86 kB 00:00 2026-02-13T02:19:06.512 INFO:teuthology.orchestra.run.trial027.stdout:(157/173): python3-websocket-client-1.2.3-2.el9 4.1 MB/s | 90 kB 00:00 2026-02-13T02:19:06.515 INFO:teuthology.orchestra.run.trial028.stdout:(10/173): ceph-radosgw-20.3.0-5241.g8f8d3829.el 26 MB/s | 13 MB 00:00 2026-02-13T02:19:06.516 INFO:teuthology.orchestra.run.trial027.stdout:(158/173): python3-xmltodict-0.12.0-15.el9.noar 1.1 MB/s | 22 kB 00:00 2026-02-13T02:19:06.524 INFO:teuthology.orchestra.run.trial107.stdout: Installing : thrift-0.15.0-4.el9.x86_64 59/184 2026-02-13T02:19:06.532 INFO:teuthology.orchestra.run.trial027.stdout:(159/173): python3-zc-lockfile-2.0-10.el9.noarc 975 kB/s | 20 kB 00:00 2026-02-13T02:19:06.534 INFO:teuthology.orchestra.run.trial027.stdout:(160/173): re2-20211101-20.el9.x86_64.rpm 8.4 MB/s | 191 kB 00:00 2026-02-13T02:19:06.549 INFO:teuthology.orchestra.run.trial028.stdout:(11/173): ceph-selinux-20.3.0-5241.g8f8d3829.el 749 kB/s | 25 kB 00:00 2026-02-13T02:19:06.553 INFO:teuthology.orchestra.run.trial027.stdout:(161/173): thrift-0.15.0-4.el9.x86_64.rpm 43 MB/s | 1.6 MB 00:00 2026-02-13T02:19:06.554 INFO:teuthology.orchestra.run.trial107.stdout: Upgrading : openssl-1:3.5.5-1.el9.x86_64 60/184 2026-02-13T02:19:06.554 INFO:teuthology.orchestra.run.trial027.stdout:(162/173): yaml-cpp-0.6.3-5.el9.x86_64.rpm 5.3 MB/s | 122 kB 00:00 2026-02-13T02:19:06.560 INFO:teuthology.orchestra.run.trial107.stdout: Upgrading : iproute-6.17.0-2.el9.x86_64 61/184 2026-02-13T02:19:06.563 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 62/184 2026-02-13T02:19:06.578 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 63/184 2026-02-13T02:19:06.582 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 64/184 2026-02-13T02:19:06.587 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 65/184 2026-02-13T02:19:06.599 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 66/184 2026-02-13T02:19:06.603 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 67/184 2026-02-13T02:19:06.612 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 68/184 2026-02-13T02:19:06.616 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 69/184 2026-02-13T02:19:06.624 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 70/184 2026-02-13T02:19:06.664 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 71/184 2026-02-13T02:19:06.695 INFO:teuthology.orchestra.run.trial028.stdout:(12/173): ceph-osd-crimson-20.3.0-5241.g8f8d382 21 MB/s | 19 MB 00:00 2026-02-13T02:19:06.739 INFO:teuthology.orchestra.run.trial028.stdout:(13/173): cephfs-mirror-20.3.0-5241.g8f8d3829.e 5.3 MB/s | 243 kB 00:00 2026-02-13T02:19:06.779 INFO:teuthology.orchestra.run.trial028.stdout:(14/173): cephfs-shell-20.3.0-5241.g8f8d3829.el 692 kB/s | 27 kB 00:00 2026-02-13T02:19:06.821 INFO:teuthology.orchestra.run.trial028.stdout:(15/173): libcephfs-devel-20.3.0-5241.g8f8d3829 897 kB/s | 37 kB 00:00 2026-02-13T02:19:06.861 INFO:teuthology.orchestra.run.trial028.stdout:(16/173): libcephfs-proxy2-20.3.0-5241.g8f8d382 603 kB/s | 24 kB 00:00 2026-02-13T02:19:06.922 INFO:teuthology.orchestra.run.trial028.stdout:(17/173): libcephfs2-20.3.0-5241.g8f8d3829.el9. 15 MB/s | 909 kB 00:00 2026-02-13T02:19:06.955 INFO:teuthology.orchestra.run.trial028.stdout:(18/173): ceph-osd-classic-20.3.0-5241.g8f8d382 12 MB/s | 16 MB 00:01 2026-02-13T02:19:06.960 INFO:teuthology.orchestra.run.trial028.stdout:(19/173): libcephsqlite-20.3.0-5241.g8f8d3829.e 4.0 MB/s | 159 kB 00:00 2026-02-13T02:19:06.990 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 72/184 2026-02-13T02:19:06.996 INFO:teuthology.orchestra.run.trial028.stdout:(20/173): librados-devel-20.3.0-5241.g8f8d3829. 2.9 MB/s | 123 kB 00:00 2026-02-13T02:19:06.999 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 73/184 2026-02-13T02:19:07.003 INFO:teuthology.orchestra.run.trial028.stdout:(21/173): libradosstriper1-20.3.0-5241.g8f8d382 5.6 MB/s | 248 kB 00:00 2026-02-13T02:19:07.004 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 74/184 2026-02-13T02:19:07.009 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 75/184 2026-02-13T02:19:07.012 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 76/184 2026-02-13T02:19:07.018 INFO:teuthology.orchestra.run.trial107.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 77/184 2026-02-13T02:19:07.020 INFO:teuthology.orchestra.run.trial107.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 78/184 2026-02-13T02:19:07.022 INFO:teuthology.orchestra.run.trial107.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 79/184 2026-02-13T02:19:07.050 INFO:teuthology.orchestra.run.trial107.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 80/184 2026-02-13T02:19:07.053 INFO:teuthology.orchestra.run.trial028.stdout:(22/173): python3-ceph-argparse-20.3.0-5241.g8f 911 kB/s | 45 kB 00:00 2026-02-13T02:19:07.093 INFO:teuthology.orchestra.run.trial107.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 81/184 2026-02-13T02:19:07.097 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 82/184 2026-02-13T02:19:07.101 INFO:teuthology.orchestra.run.trial107.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 83/184 2026-02-13T02:19:07.104 INFO:teuthology.orchestra.run.trial028.stdout:(23/173): python3-ceph-common-20.3.0-5241.g8f8d 3.5 MB/s | 185 kB 00:00 2026-02-13T02:19:07.107 INFO:teuthology.orchestra.run.trial107.stdout: Installing : uuid-1.6.2-55.el9.x86_64 84/184 2026-02-13T02:19:07.135 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 85/184 2026-02-13T02:19:07.141 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/184 2026-02-13T02:19:07.144 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 87/184 2026-02-13T02:19:07.150 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 88/184 2026-02-13T02:19:07.154 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 89/184 2026-02-13T02:19:07.158 INFO:teuthology.orchestra.run.trial028.stdout:(24/173): python3-cephfs-20.3.0-5241.g8f8d3829. 3.7 MB/s | 204 kB 00:00 2026-02-13T02:19:07.161 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 90/184 2026-02-13T02:19:07.164 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 91/184 2026-02-13T02:19:07.192 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 92/184 2026-02-13T02:19:07.202 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 93/184 2026-02-13T02:19:07.207 INFO:teuthology.orchestra.run.trial027.stdout:(163/173): librbd1-20.3.0-5241.g8f8d3829.el9.x8 4.3 MB/s | 2.8 MB 00:00 2026-02-13T02:19:07.208 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 94/184 2026-02-13T02:19:07.210 INFO:teuthology.orchestra.run.trial028.stdout:(25/173): python3-rados-20.3.0-5241.g8f8d3829.e 6.1 MB/s | 324 kB 00:00 2026-02-13T02:19:07.215 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 95/184 2026-02-13T02:19:07.250 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 96/184 2026-02-13T02:19:07.258 INFO:teuthology.orchestra.run.trial028.stdout:(26/173): python3-rbd-20.3.0-5241.g8f8d3829.el9 6.2 MB/s | 306 kB 00:00 2026-02-13T02:19:07.260 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-devel-3.9.25-2.el9.x86_64 97/184 2026-02-13T02:19:07.303 INFO:teuthology.orchestra.run.trial028.stdout:(27/173): python3-rgw-20.3.0-5241.g8f8d3829.el9 2.2 MB/s | 100 kB 00:00 2026-02-13T02:19:07.346 INFO:teuthology.orchestra.run.trial028.stdout:(28/173): rbd-fuse-20.3.0-5241.g8f8d3829.el9.x8 1.9 MB/s | 84 kB 00:00 2026-02-13T02:19:07.483 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 98/184 2026-02-13T02:19:07.507 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 99/184 2026-02-13T02:19:07.511 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 100/184 2026-02-13T02:19:07.527 INFO:teuthology.orchestra.run.trial028.stdout:(29/173): rbd-mirror-20.3.0-5241.g8f8d3829.el9. 16 MB/s | 2.9 MB 00:00 2026-02-13T02:19:07.557 INFO:teuthology.orchestra.run.trial028.stdout:(30/173): librgw2-20.3.0-5241.g8f8d3829.el9.x86 12 MB/s | 6.5 MB 00:00 2026-02-13T02:19:07.563 INFO:teuthology.orchestra.run.trial028.stdout:(31/173): rbd-nbd-20.3.0-5241.g8f8d3829.el9.x86 4.8 MB/s | 172 kB 00:00 2026-02-13T02:19:07.570 INFO:teuthology.orchestra.run.trial107.stdout: Installing : openblas-0.3.29-1.el9.x86_64 101/184 2026-02-13T02:19:07.572 INFO:teuthology.orchestra.run.trial107.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 102/184 2026-02-13T02:19:07.593 INFO:teuthology.orchestra.run.trial028.stdout:(32/173): ceph-grafana-dashboards-20.3.0-5241.g 1.6 MB/s | 57 kB 00:00 2026-02-13T02:19:07.594 INFO:teuthology.orchestra.run.trial107.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 103/184 2026-02-13T02:19:07.599 INFO:teuthology.orchestra.run.trial028.stdout:(33/173): ceph-mgr-cephadm-20.3.0-5241.g8f8d382 4.9 MB/s | 183 kB 00:00 2026-02-13T02:19:07.777 INFO:teuthology.orchestra.run.trial027.stdout:(164/173): NetworkManager-1.54.3-2.el9.x86_64.r 1.9 MB/s | 2.4 MB 00:01 2026-02-13T02:19:07.921 INFO:teuthology.orchestra.run.trial107.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 104/184 2026-02-13T02:19:07.948 INFO:teuthology.orchestra.run.trial027.stdout:(165/173): NetworkManager-team-1.54.3-2.el9.x86 217 kB/s | 37 kB 00:00 2026-02-13T02:19:07.994 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 105/184 2026-02-13T02:19:08.050 INFO:teuthology.orchestra.run.trial028.stdout:(34/173): ceph-mgr-diskprediction-local-20.3.0- 16 MB/s | 7.4 MB 00:00 2026-02-13T02:19:08.106 INFO:teuthology.orchestra.run.trial028.stdout:(35/173): ceph-mgr-modules-core-20.3.0-5241.g8f 5.2 MB/s | 297 kB 00:00 2026-02-13T02:19:08.117 INFO:teuthology.orchestra.run.trial027.stdout:(166/173): librados2-20.3.0-5241.g8f8d3829.el9. 2.2 MB/s | 3.5 MB 00:01 2026-02-13T02:19:08.152 INFO:teuthology.orchestra.run.trial028.stdout:(36/173): ceph-mgr-rook-20.3.0-5241.g8f8d3829.e 1.1 MB/s | 50 kB 00:00 2026-02-13T02:19:08.195 INFO:teuthology.orchestra.run.trial028.stdout:(37/173): ceph-prometheus-alerts-20.3.0-5241.g8 407 kB/s | 17 kB 00:00 2026-02-13T02:19:08.244 INFO:teuthology.orchestra.run.trial028.stdout:(38/173): ceph-volume-20.3.0-5241.g8f8d3829.el9 6.0 MB/s | 299 kB 00:00 2026-02-13T02:19:08.309 INFO:teuthology.orchestra.run.trial028.stdout:(39/173): cephadm-20.3.0-5241.g8f8d3829.el9.noa 16 MB/s | 1.0 MB 00:00 2026-02-13T02:19:08.338 INFO:teuthology.orchestra.run.trial027.stdout:(167/173): NetworkManager-libnm-1.54.3-2.el9.x8 1.7 MB/s | 1.9 MB 00:01 2026-02-13T02:19:08.352 INFO:teuthology.orchestra.run.trial028.stdout:(40/173): cephfs-top-20.3.0-5241.g8f8d3829.el9. 425 kB/s | 18 kB 00:00 2026-02-13T02:19:08.405 INFO:teuthology.orchestra.run.trial028.stdout:(41/173): cryptsetup-2.8.1-2.el9.x86_64.rpm 6.5 MB/s | 350 kB 00:00 2026-02-13T02:19:08.437 INFO:teuthology.orchestra.run.trial028.stdout:(42/173): hwloc-libs-2.4.1-6.el9.x86_64.rpm 66 MB/s | 2.1 MB 00:00 2026-02-13T02:19:08.446 INFO:teuthology.orchestra.run.trial027.stdout:(168/173): NetworkManager-tui-1.54.3-2.el9.x86_ 496 kB/s | 248 kB 00:00 2026-02-13T02:19:08.447 INFO:teuthology.orchestra.run.trial028.stdout:(43/173): ledmon-libs-1.1.0-3.el9.x86_64.rpm 4.1 MB/s | 40 kB 00:00 2026-02-13T02:19:08.457 INFO:teuthology.orchestra.run.trial028.stdout:(44/173): libatomic-11.5.0-14.el9.x86_64.rpm 2.3 MB/s | 23 kB 00:00 2026-02-13T02:19:08.467 INFO:teuthology.orchestra.run.trial028.stdout:(45/173): libconfig-1.7.2-9.el9.x86_64.rpm 6.7 MB/s | 72 kB 00:00 2026-02-13T02:19:08.482 INFO:teuthology.orchestra.run.trial028.stdout:(46/173): libgfortran-11.5.0-14.el9.x86_64.rpm 52 MB/s | 794 kB 00:00 2026-02-13T02:19:08.494 INFO:teuthology.orchestra.run.trial028.stdout:(47/173): libquadmath-11.5.0-14.el9.x86_64.rpm 15 MB/s | 184 kB 00:00 2026-02-13T02:19:08.504 INFO:teuthology.orchestra.run.trial028.stdout:(48/173): lksctp-tools-1.0.19-2.el9.x86_64.rpm 9.9 MB/s | 94 kB 00:00 2026-02-13T02:19:08.512 INFO:teuthology.orchestra.run.trial028.stdout:(49/173): mailcap-2.1.49-5.el9.noarch.rpm 4.0 MB/s | 33 kB 00:00 2026-02-13T02:19:08.522 INFO:teuthology.orchestra.run.trial028.stdout:(50/173): pciutils-3.7.0-7.el9.x86_64.rpm 9.3 MB/s | 93 kB 00:00 2026-02-13T02:19:08.535 INFO:teuthology.orchestra.run.trial028.stdout:(51/173): python3-cffi-1.14.5-5.el9.x86_64.rpm 19 MB/s | 253 kB 00:00 2026-02-13T02:19:08.549 INFO:teuthology.orchestra.run.trial028.stdout:(52/173): python3-cryptography-36.0.1-5.el9.x86 89 MB/s | 1.2 MB 00:00 2026-02-13T02:19:08.561 INFO:teuthology.orchestra.run.trial028.stdout:(53/173): python3-ply-3.11-14.el9.noarch.rpm 8.7 MB/s | 106 kB 00:00 2026-02-13T02:19:08.574 INFO:teuthology.orchestra.run.trial028.stdout:(54/173): python3-pycparser-2.20-6.el9.noarch.r 11 MB/s | 135 kB 00:00 2026-02-13T02:19:08.584 INFO:teuthology.orchestra.run.trial028.stdout:(55/173): python3-requests-2.25.1-10.el9.noarch 12 MB/s | 126 kB 00:00 2026-02-13T02:19:08.597 INFO:teuthology.orchestra.run.trial028.stdout:(56/173): python3-urllib3-1.26.5-7.el9.noarch.r 16 MB/s | 218 kB 00:00 2026-02-13T02:19:08.609 INFO:teuthology.orchestra.run.trial028.stdout:(57/173): unzip-6.0-59.el9.x86_64.rpm 16 MB/s | 182 kB 00:00 2026-02-13T02:19:08.622 INFO:teuthology.orchestra.run.trial028.stdout:(58/173): zip-3.0-35.el9.x86_64.rpm 19 MB/s | 266 kB 00:00 2026-02-13T02:19:08.635 INFO:teuthology.orchestra.run.trial027.stdout:(169/173): iproute-6.17.0-2.el9.x86_64.rpm 1.6 MB/s | 866 kB 00:00 2026-02-13T02:19:08.658 INFO:teuthology.orchestra.run.trial028.stdout:(59/173): automake-1.16.2-8.el9.noarch.rpm 19 MB/s | 689 kB 00:00 2026-02-13T02:19:08.668 INFO:teuthology.orchestra.run.trial028.stdout:(60/173): bison-3.7.4-5.el9.x86_64.rpm 87 MB/s | 943 kB 00:00 2026-02-13T02:19:08.673 INFO:teuthology.orchestra.run.trial028.stdout:(61/173): boost-program-options-1.75.0-13.el9.x 20 MB/s | 104 kB 00:00 2026-02-13T02:19:08.674 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 106/184 2026-02-13T02:19:08.678 INFO:teuthology.orchestra.run.trial028.stdout:(62/173): elfutils-libelf-devel-0.194-1.el9.x86 17 MB/s | 79 kB 00:00 2026-02-13T02:19:08.684 INFO:teuthology.orchestra.run.trial028.stdout:(63/173): flex-2.6.4-9.el9.x86_64.rpm 49 MB/s | 313 kB 00:00 2026-02-13T02:19:08.688 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 107/184 2026-02-13T02:19:08.689 INFO:teuthology.orchestra.run.trial028.stdout:(64/173): flexiblas-3.0.4-8.el9.x86_64.rpm 7.8 MB/s | 33 kB 00:00 2026-02-13T02:19:08.692 INFO:teuthology.orchestra.run.trial107.stdout: Installing : libzstd-devel-1.5.5-1.el9.x86_64 108/184 2026-02-13T02:19:08.694 INFO:teuthology.orchestra.run.trial107.stdout: Installing : libuuid-devel-2.37.4-21.el9.x86_64 109/184 2026-02-13T02:19:08.700 INFO:teuthology.orchestra.run.trial107.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 110/184 2026-02-13T02:19:08.728 INFO:teuthology.orchestra.run.trial028.stdout:(65/173): ceph-mgr-dashboard-20.3.0-5241.g8f8d3 13 MB/s | 15 MB 00:01 2026-02-13T02:19:08.738 INFO:teuthology.orchestra.run.trial028.stdout:(66/173): flexiblas-openblas-openmp-3.0.4-8.el9 1.7 MB/s | 18 kB 00:00 2026-02-13T02:19:08.742 INFO:teuthology.orchestra.run.trial028.stdout:(67/173): libaio-devel-0.3.111-13.el9.x86_64.rp 2.8 MB/s | 11 kB 00:00 2026-02-13T02:19:08.751 INFO:teuthology.orchestra.run.trial028.stdout:(68/173): flexiblas-netlib-3.0.4-8.el9.x86_64.r 48 MB/s | 3.0 MB 00:00 2026-02-13T02:19:08.755 INFO:teuthology.orchestra.run.trial028.stdout:(69/173): libnbd-1.20.3-4.el9.x86_64.rpm 13 MB/s | 164 kB 00:00 2026-02-13T02:19:08.757 INFO:teuthology.orchestra.run.trial028.stdout:(70/173): libpmemobj-1.12.1-1.el9.x86_64.rpm 27 MB/s | 160 kB 00:00 2026-02-13T02:19:08.760 INFO:teuthology.orchestra.run.trial028.stdout:(71/173): librabbitmq-0.11.0-7.el9.x86_64.rpm 9.9 MB/s | 45 kB 00:00 2026-02-13T02:19:08.867 INFO:teuthology.orchestra.run.trial028.stdout:(72/173): ceph-test-20.3.0-5241.g8f8d3829.el9.x 34 MB/s | 78 MB 00:02 2026-02-13T02:19:08.868 INFO:teuthology.orchestra.run.trial028.stdout:(73/173): libstoragemgmt-1.10.1-1.el9.x86_64.rp 2.2 MB/s | 246 kB 00:00 2026-02-13T02:19:08.870 INFO:teuthology.orchestra.run.trial028.stdout:(74/173): librdkafka-1.6.1-102.el9.x86_64.rpm 5.7 MB/s | 662 kB 00:00 2026-02-13T02:19:08.873 INFO:teuthology.orchestra.run.trial028.stdout:(75/173): libuuid-devel-2.37.4-21.el9.x86_64.rp 5.5 MB/s | 25 kB 00:00 2026-02-13T02:19:08.876 INFO:teuthology.orchestra.run.trial028.stdout:(76/173): libxslt-1.1.34-12.el9.x86_64.rpm 37 MB/s | 233 kB 00:00 2026-02-13T02:19:08.877 INFO:teuthology.orchestra.run.trial028.stdout:(77/173): libzstd-devel-1.5.5-1.el9.x86_64.rpm 11 MB/s | 51 kB 00:00 2026-02-13T02:19:08.884 INFO:teuthology.orchestra.run.trial028.stdout:(78/173): lua-5.4.4-4.el9.x86_64.rpm 26 MB/s | 188 kB 00:00 2026-02-13T02:19:08.885 INFO:teuthology.orchestra.run.trial028.stdout:(79/173): lttng-ust-2.12.0-6.el9.x86_64.rpm 31 MB/s | 292 kB 00:00 2026-02-13T02:19:08.889 INFO:teuthology.orchestra.run.trial028.stdout:(80/173): openblas-0.3.29-1.el9.x86_64.rpm 9.3 MB/s | 42 kB 00:00 2026-02-13T02:19:08.891 INFO:teuthology.orchestra.run.trial028.stdout:(81/173): libtool-2.4.6-46.el9.x86_64.rpm 24 MB/s | 581 kB 00:00 2026-02-13T02:19:08.895 INFO:teuthology.orchestra.run.trial028.stdout:(82/173): perl-Thread-Queue-3.14-460.el9.noarch 5.3 MB/s | 22 kB 00:00 2026-02-13T02:19:08.900 INFO:teuthology.orchestra.run.trial028.stdout:(83/173): perl-threads-2.25-460.el9.x86_64.rpm 12 MB/s | 58 kB 00:00 2026-02-13T02:19:08.907 INFO:teuthology.orchestra.run.trial028.stdout:(84/173): openblas-openmp-0.3.29-1.el9.x86_64.r 243 MB/s | 5.3 MB 00:00 2026-02-13T02:19:08.907 INFO:teuthology.orchestra.run.trial028.stdout:(85/173): perl-threads-shared-1.61-460.el9.x86_ 6.2 MB/s | 45 kB 00:00 2026-02-13T02:19:08.921 INFO:teuthology.orchestra.run.trial028.stdout:(86/173): postgresql-13.23-1.el9.x86_64.rpm 117 MB/s | 1.6 MB 00:00 2026-02-13T02:19:08.927 INFO:teuthology.orchestra.run.trial028.stdout:(87/173): postgresql-private-libs-13.23-1.el9.x 24 MB/s | 138 kB 00:00 2026-02-13T02:19:08.937 INFO:teuthology.orchestra.run.trial028.stdout:(88/173): openssl-devel-3.5.5-1.el9.x86_64.rpm 101 MB/s | 4.8 MB 00:00 2026-02-13T02:19:08.946 INFO:teuthology.orchestra.run.trial028.stdout:(89/173): protobuf-3.14.0-17.el9.x86_64.rpm 108 MB/s | 1.0 MB 00:00 2026-02-13T02:19:08.954 INFO:teuthology.orchestra.run.trial028.stdout:(90/173): postgresql-server-13.23-1.el9.x86_64. 216 MB/s | 5.8 MB 00:00 2026-02-13T02:19:08.955 INFO:teuthology.orchestra.run.trial028.stdout:(91/173): postgresql-contrib-13.23-1.el9.x86_64 19 MB/s | 903 kB 00:00 2026-02-13T02:19:08.959 INFO:teuthology.orchestra.run.trial028.stdout:(92/173): python3-colorama-0.4.6-3.el9.noarch.r 11 MB/s | 61 kB 00:00 2026-02-13T02:19:08.963 INFO:teuthology.orchestra.run.trial028.stdout:(93/173): python3-devel-3.9.25-2.el9.x86_64.rpm 32 MB/s | 244 kB 00:00 2026-02-13T02:19:08.966 INFO:teuthology.orchestra.run.trial028.stdout:(94/173): python3-jinja2-2.11.3-8.el9.noarch.rp 38 MB/s | 249 kB 00:00 2026-02-13T02:19:08.969 INFO:teuthology.orchestra.run.trial027.stdout:(170/173): iproute-tc-6.17.0-2.el9.x86_64.rpm 719 kB/s | 454 kB 00:00 2026-02-13T02:19:08.974 INFO:teuthology.orchestra.run.trial028.stdout:(95/173): python3-babel-2.9.1-2.el9.noarch.rpm 214 MB/s | 6.0 MB 00:00 2026-02-13T02:19:08.974 INFO:teuthology.orchestra.run.trial028.stdout:(96/173): python3-jmespath-1.0.1-1.el9.noarch.r 4.0 MB/s | 48 kB 00:00 2026-02-13T02:19:08.975 INFO:teuthology.orchestra.run.trial028.stdout:(97/173): python3-libstoragemgmt-1.10.1-1.el9.x 19 MB/s | 177 kB 00:00 2026-02-13T02:19:08.979 INFO:teuthology.orchestra.run.trial028.stdout:(98/173): python3-markupsafe-1.1.1-12.el9.x86_6 6.9 MB/s | 35 kB 00:00 2026-02-13T02:19:08.980 INFO:teuthology.orchestra.run.trial027.stdout:(171/173): openssl-3.5.5-1.el9.x86_64.rpm 2.8 MB/s | 1.5 MB 00:00 2026-02-13T02:19:08.983 INFO:teuthology.orchestra.run.trial028.stdout:(99/173): python3-numpy-f2py-1.23.5-2.el9.x86_6 57 MB/s | 442 kB 00:00 2026-02-13T02:19:08.985 INFO:teuthology.orchestra.run.trial028.stdout:(100/173): python3-packaging-20.9-5.el9.noarch. 14 MB/s | 77 kB 00:00 2026-02-13T02:19:08.985 INFO:teuthology.orchestra.run.trial107.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 111/184 2026-02-13T02:19:08.986 INFO:teuthology.orchestra.run.trial107.stdout: Installing : librgw2-2:20.3.0-5241.g8f8d3829.el9.x86_64 112/184 2026-02-13T02:19:08.990 INFO:teuthology.orchestra.run.trial028.stdout:(101/173): python3-pyasn1-0.4.8-6.el9.noarch.rp 30 MB/s | 159 kB 00:00 2026-02-13T02:19:08.990 INFO:teuthology.orchestra.run.trial028.stdout:(102/173): python3-protobuf-3.14.0-17.el9.noarc 35 MB/s | 267 kB 00:00 2026-02-13T02:19:08.996 INFO:teuthology.orchestra.run.trial028.stdout:(103/173): python3-requests-oauthlib-1.3.0-12.e 10 MB/s | 54 kB 00:00 2026-02-13T02:19:08.997 INFO:teuthology.orchestra.run.trial028.stdout:(104/173): python3-pyasn1-modules-0.4.8-6.el9.n 40 MB/s | 279 kB 00:00 2026-02-13T02:19:09.005 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: librgw2-2:20.3.0-5241.g8f8d3829.el9.x86_64 112/184 2026-02-13T02:19:09.006 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-rgw-2:20.3.0-5241.g8f8d3829.el9.x86_64 113/184 2026-02-13T02:19:09.009 INFO:teuthology.orchestra.run.trial028.stdout:(105/173): python3-numpy-1.23.5-2.el9.x86_64.rp 179 MB/s | 6.1 MB 00:00 2026-02-13T02:19:09.009 INFO:teuthology.orchestra.run.trial028.stdout:(106/173): python3-toml-0.10.2-6.el9.noarch.rpm 3.3 MB/s | 42 kB 00:00 2026-02-13T02:19:09.015 INFO:teuthology.orchestra.run.trial028.stdout:(107/173): python3-wcwidth-0.2.5-8.el9.noarch.r 7.6 MB/s | 44 kB 00:00 2026-02-13T02:19:09.016 INFO:teuthology.orchestra.run.trial028.stdout:(108/173): qatlib-25.08.0-2.el9.x86_64.rpm 37 MB/s | 240 kB 00:00 2026-02-13T02:19:09.019 INFO:teuthology.orchestra.run.trial028.stdout:(109/173): qatlib-service-25.08.0-2.el9.x86_64. 7.7 MB/s | 37 kB 00:00 2026-02-13T02:19:09.020 INFO:teuthology.orchestra.run.trial028.stdout:(110/173): qatzip-libs-1.3.1-1.el9.x86_64.rpm 14 MB/s | 66 kB 00:00 2026-02-13T02:19:09.025 INFO:teuthology.orchestra.run.trial028.stdout:(111/173): uuid-1.6.2-55.el9.x86_64.rpm 13 MB/s | 58 kB 00:00 2026-02-13T02:19:09.026 INFO:teuthology.orchestra.run.trial028.stdout:(112/173): socat-1.7.4.1-8.el9.x86_64.rpm 46 MB/s | 303 kB 00:00 2026-02-13T02:19:09.029 INFO:teuthology.orchestra.run.trial028.stdout:(113/173): xfsprogs-devel-6.4.0-7.el9.x86_64.rp 21 MB/s | 96 kB 00:00 2026-02-13T02:19:09.031 INFO:teuthology.orchestra.run.trial028.stdout:(114/173): xmlstarlet-1.6.1-20.el9.x86_64.rpm 13 MB/s | 64 kB 00:00 2026-02-13T02:19:09.035 INFO:teuthology.orchestra.run.trial028.stdout:(115/173): zlib-devel-1.2.11-41.el9.x86_64.rpm 8.6 MB/s | 45 kB 00:00 2026-02-13T02:19:09.075 INFO:teuthology.orchestra.run.trial028.stdout:(116/173): python3-scipy-1.9.3-2.el9.x86_64.rpm 243 MB/s | 19 MB 00:00 2026-02-13T02:19:09.140 INFO:teuthology.orchestra.run.trial028.stdout:(117/173): protobuf-compiler-3.14.0-17.el9.x86_ 8.0 MB/s | 862 kB 00:00 2026-02-13T02:19:09.141 INFO:teuthology.orchestra.run.trial028.stdout:(118/173): abseil-cpp-20211102.0-4.el9.x86_64.r 8.3 MB/s | 551 kB 00:00 2026-02-13T02:19:09.149 INFO:teuthology.orchestra.run.trial028.stdout:(119/173): grpc-data-1.46.7-10.el9.noarch.rpm 2.4 MB/s | 19 kB 00:00 2026-02-13T02:19:09.181 INFO:teuthology.orchestra.run.trial028.stdout:(120/173): gperftools-libs-2.9.1-3.el9.x86_64.r 7.3 MB/s | 308 kB 00:00 2026-02-13T02:19:09.195 INFO:teuthology.orchestra.run.trial028.stdout:(121/173): libarrow-9.0.0-15.el9.x86_64.rpm 97 MB/s | 4.4 MB 00:00 2026-02-13T02:19:09.195 INFO:teuthology.orchestra.run.trial028.stdout:(122/173): libarrow-doc-9.0.0-15.el9.noarch.rpm 1.8 MB/s | 25 kB 00:00 2026-02-13T02:19:09.204 INFO:teuthology.orchestra.run.trial028.stdout:(123/173): liboath-2.6.12-1.el9.x86_64.rpm 5.1 MB/s | 49 kB 00:00 2026-02-13T02:19:09.204 INFO:teuthology.orchestra.run.trial027.stdout:(172/173): openssl-fips-provider-3.5.5-1.el9.x8 1.4 MB/s | 814 kB 00:00 2026-02-13T02:19:09.205 INFO:teuthology.orchestra.run.trial028.stdout:(124/173): libunwind-1.6.2-1.el9.x86_64.rpm 7.0 MB/s | 67 kB 00:00 2026-02-13T02:19:09.213 INFO:teuthology.orchestra.run.trial028.stdout:(125/173): luarocks-3.9.2-5.el9.noarch.rpm 17 MB/s | 151 kB 00:00 2026-02-13T02:19:09.222 INFO:teuthology.orchestra.run.trial028.stdout:(126/173): parquet-libs-9.0.0-15.el9.x86_64.rpm 47 MB/s | 838 kB 00:00 2026-02-13T02:19:09.231 INFO:teuthology.orchestra.run.trial028.stdout:(127/173): python3-asyncssh-2.13.2-5.el9.noarch 30 MB/s | 548 kB 00:00 2026-02-13T02:19:09.231 INFO:teuthology.orchestra.run.trial028.stdout:(128/173): python3-autocommand-2.2.2-8.el9.noar 3.2 MB/s | 29 kB 00:00 2026-02-13T02:19:09.239 INFO:teuthology.orchestra.run.trial028.stdout:(129/173): python3-backports-tarfile-1.2.0-1.el 7.0 MB/s | 60 kB 00:00 2026-02-13T02:19:09.240 INFO:teuthology.orchestra.run.trial028.stdout:(130/173): python3-bcrypt-3.2.2-1.el9.x86_64.rp 5.1 MB/s | 43 kB 00:00 2026-02-13T02:19:09.240 INFO:teuthology.orchestra.run.trial028.stdout:(131/173): lua-devel-5.4.4-4.el9.x86_64.rpm 106 kB/s | 22 kB 00:00 2026-02-13T02:19:09.249 INFO:teuthology.orchestra.run.trial028.stdout:(132/173): python3-certifi-2023.05.07-4.el9.noa 1.5 MB/s | 14 kB 00:00 2026-02-13T02:19:09.251 INFO:teuthology.orchestra.run.trial028.stdout:(133/173): python3-cachetools-4.2.4-1.el9.noarc 2.7 MB/s | 32 kB 00:00 2026-02-13T02:19:09.259 INFO:teuthology.orchestra.run.trial028.stdout:(134/173): python3-cherrypy-18.6.1-2.el9.noarch 35 MB/s | 358 kB 00:00 2026-02-13T02:19:09.264 INFO:teuthology.orchestra.run.trial028.stdout:(135/173): python3-cmd2-2.4.2-2.el9.noarch.rpm 24 MB/s | 303 kB 00:00 2026-02-13T02:19:09.271 INFO:teuthology.orchestra.run.trial028.stdout:(136/173): python3-google-auth-2.45.0-1.el9.noa 21 MB/s | 254 kB 00:00 2026-02-13T02:19:09.274 INFO:teuthology.orchestra.run.trial028.stdout:(137/173): python3-cheroot-10.0.1-4.el9.noarch. 5.0 MB/s | 173 kB 00:00 2026-02-13T02:19:09.286 INFO:teuthology.orchestra.run.trial028.stdout:(138/173): python3-grpcio-tools-1.46.7-10.el9.x 9.2 MB/s | 144 kB 00:00 2026-02-13T02:19:09.287 INFO:teuthology.orchestra.run.trial028.stdout:(139/173): python3-jaraco-8.2.1-3.el9.noarch.rp 854 kB/s | 11 kB 00:00 2026-02-13T02:19:09.295 INFO:teuthology.orchestra.run.trial028.stdout:(140/173): python3-grpcio-1.46.7-10.el9.x86_64. 66 MB/s | 2.0 MB 00:00 2026-02-13T02:19:09.296 INFO:teuthology.orchestra.run.trial028.stdout:(141/173): python3-jaraco-classes-3.2.1-5.el9.n 1.9 MB/s | 18 kB 00:00 2026-02-13T02:19:09.296 INFO:teuthology.orchestra.run.trial028.stdout:(142/173): python3-jaraco-collections-3.0.0-8.e 2.6 MB/s | 23 kB 00:00 2026-02-13T02:19:09.303 INFO:teuthology.orchestra.run.trial028.stdout:(143/173): python3-jaraco-context-6.0.1-3.el9.n 2.5 MB/s | 20 kB 00:00 2026-02-13T02:19:09.304 INFO:teuthology.orchestra.run.trial028.stdout:(144/173): python3-jaraco-functools-3.5.0-2.el9 2.5 MB/s | 19 kB 00:00 2026-02-13T02:19:09.304 INFO:teuthology.orchestra.run.trial028.stdout:(145/173): python3-jaraco-text-4.0.0-2.el9.noar 3.2 MB/s | 26 kB 00:00 2026-02-13T02:19:09.317 INFO:teuthology.orchestra.run.trial028.stdout:(146/173): python3-natsort-7.1.1-5.el9.noarch.r 4.5 MB/s | 58 kB 00:00 2026-02-13T02:19:09.317 INFO:teuthology.orchestra.run.trial028.stdout:(147/173): python3-more-itertools-8.12.0-2.el9. 5.8 MB/s | 79 kB 00:00 2026-02-13T02:19:09.320 INFO:teuthology.orchestra.run.trial028.stdout:(148/173): python3-kubernetes-26.1.0-3.el9.noar 59 MB/s | 1.0 MB 00:00 2026-02-13T02:19:09.324 INFO:teuthology.orchestra.run.trial028.stdout:(149/173): python3-portend-3.1.0-2.el9.noarch.r 2.1 MB/s | 16 kB 00:00 2026-02-13T02:19:09.326 INFO:teuthology.orchestra.run.trial028.stdout:(150/173): python3-pyOpenSSL-21.0.0-1.el9.noarc 10 MB/s | 90 kB 00:00 2026-02-13T02:19:09.330 INFO:teuthology.orchestra.run.trial028.stdout:(151/173): python3-pyperclip-1.8.2-2.el9.noarch 3.0 MB/s | 28 kB 00:00 2026-02-13T02:19:09.333 INFO:teuthology.orchestra.run.trial028.stdout:(152/173): python3-repoze-lru-0.7-16.el9.noarch 3.5 MB/s | 31 kB 00:00 2026-02-13T02:19:09.337 INFO:teuthology.orchestra.run.trial028.stdout:(153/173): python3-routes-2.5.1-5.el9.noarch.rp 16 MB/s | 188 kB 00:00 2026-02-13T02:19:09.340 INFO:teuthology.orchestra.run.trial028.stdout:(154/173): python3-rsa-4.9-2.el9.noarch.rpm 5.8 MB/s | 59 kB 00:00 2026-02-13T02:19:09.341 INFO:teuthology.orchestra.run.trial028.stdout:(155/173): python3-tempora-5.0.0-2.el9.noarch.r 4.4 MB/s | 36 kB 00:00 2026-02-13T02:19:09.348 INFO:teuthology.orchestra.run.trial028.stdout:(156/173): python3-typing-extensions-4.15.0-1.e 8.5 MB/s | 86 kB 00:00 2026-02-13T02:19:09.351 INFO:teuthology.orchestra.run.trial028.stdout:(157/173): python3-websocket-client-1.2.3-2.el9 8.2 MB/s | 90 kB 00:00 2026-02-13T02:19:09.351 INFO:teuthology.orchestra.run.trial028.stdout:(158/173): python3-xmltodict-0.12.0-15.el9.noar 2.1 MB/s | 22 kB 00:00 2026-02-13T02:19:09.355 INFO:teuthology.orchestra.run.trial028.stdout:(159/173): python3-zc-lockfile-2.0-10.el9.noarc 2.6 MB/s | 20 kB 00:00 2026-02-13T02:19:09.361 INFO:teuthology.orchestra.run.trial028.stdout:(160/173): re2-20211101-20.el9.x86_64.rpm 18 MB/s | 191 kB 00:00 2026-02-13T02:19:09.367 INFO:teuthology.orchestra.run.trial028.stdout:(161/173): yaml-cpp-0.6.3-5.el9.x86_64.rpm 10 MB/s | 122 kB 00:00 2026-02-13T02:19:09.368 INFO:teuthology.orchestra.run.trial027.stdout:(173/173): openssl-libs-3.5.5-1.el9.x86_64.rpm 5.8 MB/s | 2.3 MB 00:00 2026-02-13T02:19:09.372 INFO:teuthology.orchestra.run.trial027.stdout:-------------------------------------------------------------------------------- 2026-02-13T02:19:09.372 INFO:teuthology.orchestra.run.trial027.stdout:Total 11 MB/s | 302 MB 00:27 2026-02-13T02:19:09.385 INFO:teuthology.orchestra.run.trial028.stdout:(162/173): thrift-0.15.0-4.el9.x86_64.rpm 48 MB/s | 1.6 MB 00:00 2026-02-13T02:19:09.447 INFO:teuthology.orchestra.run.trial028.stdout:(163/173): NetworkManager-1.54.3-2.el9.x86_64.r 38 MB/s | 2.4 MB 00:00 2026-02-13T02:19:09.465 INFO:teuthology.orchestra.run.trial028.stdout:(164/173): NetworkManager-libnm-1.54.3-2.el9.x8 113 MB/s | 1.9 MB 00:00 2026-02-13T02:19:09.474 INFO:teuthology.orchestra.run.trial028.stdout:(165/173): NetworkManager-team-1.54.3-2.el9.x86 4.0 MB/s | 37 kB 00:00 2026-02-13T02:19:09.480 INFO:teuthology.orchestra.run.trial028.stdout:(166/173): librbd1-20.3.0-5241.g8f8d3829.el9.x8 25 MB/s | 2.8 MB 00:00 2026-02-13T02:19:09.485 INFO:teuthology.orchestra.run.trial028.stdout:(167/173): librados2-20.3.0-5241.g8f8d3829.el9. 28 MB/s | 3.5 MB 00:00 2026-02-13T02:19:09.486 INFO:teuthology.orchestra.run.trial028.stdout:(168/173): NetworkManager-tui-1.54.3-2.el9.x86_ 20 MB/s | 248 kB 00:00 2026-02-13T02:19:09.500 INFO:teuthology.orchestra.run.trial028.stdout:(169/173): openssl-3.5.5-1.el9.x86_64.rpm 105 MB/s | 1.5 MB 00:00 2026-02-13T02:19:09.512 INFO:teuthology.orchestra.run.trial028.stdout:(170/173): openssl-fips-provider-3.5.5-1.el9.x8 66 MB/s | 814 kB 00:00 2026-02-13T02:19:09.529 INFO:teuthology.orchestra.run.trial028.stdout:(171/173): openssl-libs-3.5.5-1.el9.x86_64.rpm 141 MB/s | 2.3 MB 00:00 2026-02-13T02:19:09.539 INFO:teuthology.orchestra.run.trial028.stdout:(172/173): iproute-tc-6.17.0-2.el9.x86_64.rpm 8.2 MB/s | 454 kB 00:00 2026-02-13T02:19:09.541 INFO:teuthology.orchestra.run.trial028.stdout:(173/173): iproute-6.17.0-2.el9.x86_64.rpm 14 MB/s | 866 kB 00:00 2026-02-13T02:19:09.544 INFO:teuthology.orchestra.run.trial028.stdout:-------------------------------------------------------------------------------- 2026-02-13T02:19:09.544 INFO:teuthology.orchestra.run.trial028.stdout:Total 64 MB/s | 302 MB 00:04 2026-02-13T02:19:10.117 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: ceph-common-2:20.3.0-5241.g8f8d3829.el9.x86_64 114/184 2026-02-13T02:19:10.120 INFO:teuthology.orchestra.run.trial107.stdout: Installing : ceph-common-2:20.3.0-5241.g8f8d3829.el9.x86_64 114/184 2026-02-13T02:19:10.129 INFO:teuthology.orchestra.run.trial027.stdout:Running transaction check 2026-02-13T02:19:10.142 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: ceph-common-2:20.3.0-5241.g8f8d3829.el9.x86_64 114/184 2026-02-13T02:19:10.156 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/184 2026-02-13T02:19:10.172 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/184 2026-02-13T02:19:10.200 INFO:teuthology.orchestra.run.trial027.stdout:Transaction check succeeded. 2026-02-13T02:19:10.200 INFO:teuthology.orchestra.run.trial027.stdout:Running transaction test 2026-02-13T02:19:10.248 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/184 2026-02-13T02:19:10.257 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 118/184 2026-02-13T02:19:10.280 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/184 2026-02-13T02:19:10.310 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 120/184 2026-02-13T02:19:10.358 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction check 2026-02-13T02:19:10.361 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/184 2026-02-13T02:19:10.367 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/184 2026-02-13T02:19:10.370 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/184 2026-02-13T02:19:10.374 INFO:teuthology.orchestra.run.trial107.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 124/184 2026-02-13T02:19:10.377 INFO:teuthology.orchestra.run.trial107.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 125/184 2026-02-13T02:19:10.378 INFO:teuthology.orchestra.run.trial107.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 126/184 2026-02-13T02:19:10.396 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 126/184 2026-02-13T02:19:10.428 INFO:teuthology.orchestra.run.trial028.stdout:Transaction check succeeded. 2026-02-13T02:19:10.428 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction test 2026-02-13T02:19:10.649 INFO:teuthology.orchestra.run.trial107.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 127/184 2026-02-13T02:19:10.652 INFO:teuthology.orchestra.run.trial107.stdout: Installing : ceph-base-2:20.3.0-5241.g8f8d3829.el9.x86_64 128/184 2026-02-13T02:19:10.689 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: ceph-base-2:20.3.0-5241.g8f8d3829.el9.x86_64 128/184 2026-02-13T02:19:10.689 INFO:teuthology.orchestra.run.trial107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-13T02:19:10.689 INFO:teuthology.orchestra.run.trial107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-13T02:19:10.690 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:19:10.693 INFO:teuthology.orchestra.run.trial107.stdout: Installing : ceph-selinux-2:20.3.0-5241.g8f8d3829.el9.x86_64 129/184 2026-02-13T02:19:11.298 INFO:teuthology.orchestra.run.trial027.stdout:Transaction test succeeded. 2026-02-13T02:19:11.298 INFO:teuthology.orchestra.run.trial027.stdout:Running transaction 2026-02-13T02:19:11.516 INFO:teuthology.orchestra.run.trial028.stdout:Transaction test succeeded. 2026-02-13T02:19:11.517 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction 2026-02-13T02:19:12.467 INFO:teuthology.orchestra.run.trial027.stdout: Preparing : 1/1 2026-02-13T02:19:12.470 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/184 2026-02-13T02:19:12.478 INFO:teuthology.orchestra.run.trial027.stdout: Installing : liboath-2.6.12-1.el9.x86_64 2/184 2026-02-13T02:19:12.499 INFO:teuthology.orchestra.run.trial027.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/184 2026-02-13T02:19:12.518 INFO:teuthology.orchestra.run.trial027.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/184 2026-02-13T02:19:12.520 INFO:teuthology.orchestra.run.trial027.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 5/184 2026-02-13T02:19:12.525 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/184 2026-02-13T02:19:12.565 INFO:teuthology.orchestra.run.trial027.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 7/184 2026-02-13T02:19:12.570 INFO:teuthology.orchestra.run.trial027.stdout: Upgrading : NetworkManager-libnm-1:1.54.3-2.el9.x86_64 8/184 2026-02-13T02:19:12.605 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: NetworkManager-1:1.54.3-2.el9.x86_64 9/184 2026-02-13T02:19:12.607 INFO:teuthology.orchestra.run.trial027.stdout: Upgrading : NetworkManager-1:1.54.3-2.el9.x86_64 9/184 2026-02-13T02:19:12.672 INFO:teuthology.orchestra.run.trial028.stdout: Preparing : 1/1 2026-02-13T02:19:12.674 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/184 2026-02-13T02:19:12.683 INFO:teuthology.orchestra.run.trial028.stdout: Installing : liboath-2.6.12-1.el9.x86_64 2/184 2026-02-13T02:19:12.693 INFO:teuthology.orchestra.run.trial028.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/184 2026-02-13T02:19:12.697 INFO:teuthology.orchestra.run.trial028.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/184 2026-02-13T02:19:12.699 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 5/184 2026-02-13T02:19:12.704 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/184 2026-02-13T02:19:12.710 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: NetworkManager-1:1.54.3-2.el9.x86_64 9/184 2026-02-13T02:19:12.713 INFO:teuthology.orchestra.run.trial027.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 10/184 2026-02-13T02:19:12.745 INFO:teuthology.orchestra.run.trial027.stdout: Installing : re2-1:20211101-20.el9.x86_64 11/184 2026-02-13T02:19:12.748 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 7/184 2026-02-13T02:19:12.751 INFO:teuthology.orchestra.run.trial028.stdout: Upgrading : NetworkManager-libnm-1:1.54.3-2.el9.x86_64 8/184 2026-02-13T02:19:12.767 INFO:teuthology.orchestra.run.trial027.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 12/184 2026-02-13T02:19:12.777 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 13/184 2026-02-13T02:19:12.781 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-packaging-20.9-5.el9.noarch 14/184 2026-02-13T02:19:12.786 INFO:teuthology.orchestra.run.trial027.stdout: Installing : perl-threads-1:2.25-460.el9.x86_64 15/184 2026-02-13T02:19:12.787 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: NetworkManager-1:1.54.3-2.el9.x86_64 9/184 2026-02-13T02:19:12.790 INFO:teuthology.orchestra.run.trial028.stdout: Upgrading : NetworkManager-1:1.54.3-2.el9.x86_64 9/184 2026-02-13T02:19:12.791 INFO:teuthology.orchestra.run.trial027.stdout: Installing : lua-5.4.4-4.el9.x86_64 16/184 2026-02-13T02:19:12.794 INFO:teuthology.orchestra.run.trial027.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 17/184 2026-02-13T02:19:12.798 INFO:teuthology.orchestra.run.trial027.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 18/184 2026-02-13T02:19:12.821 INFO:teuthology.orchestra.run.trial027.stdout: Installing : unzip-6.0-59.el9.x86_64 19/184 2026-02-13T02:19:12.834 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 20/184 2026-02-13T02:19:12.839 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 21/184 2026-02-13T02:19:12.843 INFO:teuthology.orchestra.run.trial027.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 22/184 2026-02-13T02:19:12.849 INFO:teuthology.orchestra.run.trial027.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 23/184 2026-02-13T02:19:12.850 INFO:teuthology.orchestra.run.trial027.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 24/184 2026-02-13T02:19:12.852 INFO:teuthology.orchestra.run.trial027.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 25/184 2026-02-13T02:19:12.861 INFO:teuthology.orchestra.run.trial027.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 26/184 2026-02-13T02:19:12.892 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: NetworkManager-1:1.54.3-2.el9.x86_64 9/184 2026-02-13T02:19:12.896 INFO:teuthology.orchestra.run.trial027.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 27/184 2026-02-13T02:19:12.896 INFO:teuthology.orchestra.run.trial028.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 10/184 2026-02-13T02:19:12.900 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-ceph-common-2:20.3.0-5241.g8f8d3829.el9. 28/184 2026-02-13T02:19:12.902 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-ceph-argparse-2:20.3.0-5241.g8f8d3829.el 29/184 2026-02-13T02:19:12.903 INFO:teuthology.orchestra.run.trial027.stdout: Installing : libcephfs-proxy2-2:20.3.0-5241.g8f8d3829.el9.x86 30/184 2026-02-13T02:19:12.924 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5241.g8f8d3829.el9.x86 30/184 2026-02-13T02:19:12.930 INFO:teuthology.orchestra.run.trial028.stdout: Installing : re2-1:20211101-20.el9.x86_64 11/184 2026-02-13T02:19:12.930 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 31/184 2026-02-13T02:19:12.952 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 12/184 2026-02-13T02:19:12.954 INFO:teuthology.orchestra.run.trial027.stdout: Installing : zip-3.0-35.el9.x86_64 32/184 2026-02-13T02:19:12.959 INFO:teuthology.orchestra.run.trial027.stdout: Installing : luarocks-3.9.2-5.el9.noarch 33/184 2026-02-13T02:19:12.961 INFO:teuthology.orchestra.run.trial027.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 34/184 2026-02-13T02:19:12.963 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 13/184 2026-02-13T02:19:12.963 INFO:teuthology.orchestra.run.trial027.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/184 2026-02-13T02:19:12.966 INFO:teuthology.orchestra.run.trial027.stdout: Installing : perl-threads-shared-1.61-460.el9.x86_64 36/184 2026-02-13T02:19:12.967 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-packaging-20.9-5.el9.noarch 14/184 2026-02-13T02:19:12.972 INFO:teuthology.orchestra.run.trial028.stdout: Installing : perl-threads-1:2.25-460.el9.x86_64 15/184 2026-02-13T02:19:12.977 INFO:teuthology.orchestra.run.trial028.stdout: Installing : lua-5.4.4-4.el9.x86_64 16/184 2026-02-13T02:19:12.980 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 17/184 2026-02-13T02:19:12.984 INFO:teuthology.orchestra.run.trial028.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 18/184 2026-02-13T02:19:12.994 INFO:teuthology.orchestra.run.trial027.stdout: Installing : perl-Thread-Queue-3.14-460.el9.noarch 37/184 2026-02-13T02:19:13.007 INFO:teuthology.orchestra.run.trial028.stdout: Installing : unzip-6.0-59.el9.x86_64 19/184 2026-02-13T02:19:13.020 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 20/184 2026-02-13T02:19:13.026 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 21/184 2026-02-13T02:19:13.029 INFO:teuthology.orchestra.run.trial028.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 22/184 2026-02-13T02:19:13.035 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 23/184 2026-02-13T02:19:13.037 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 24/184 2026-02-13T02:19:13.038 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 25/184 2026-02-13T02:19:13.044 INFO:teuthology.orchestra.run.trial027.stdout: Installing : automake-1.16.2-8.el9.noarch 38/184 2026-02-13T02:19:13.048 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 26/184 2026-02-13T02:19:13.057 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/184 2026-02-13T02:19:13.062 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/184 2026-02-13T02:19:13.070 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 41/184 2026-02-13T02:19:13.082 INFO:teuthology.orchestra.run.trial028.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 27/184 2026-02-13T02:19:13.087 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-ceph-common-2:20.3.0-5241.g8f8d3829.el9. 28/184 2026-02-13T02:19:13.088 INFO:teuthology.orchestra.run.trial027.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 42/184 2026-02-13T02:19:13.089 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-ceph-argparse-2:20.3.0-5241.g8f8d3829.el 29/184 2026-02-13T02:19:13.090 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libcephfs-proxy2-2:20.3.0-5241.g8f8d3829.el9.x86 30/184 2026-02-13T02:19:13.094 INFO:teuthology.orchestra.run.trial027.stdout: Upgrading : openssl-libs-1:3.5.5-1.el9.x86_64 43/184 2026-02-13T02:19:13.114 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5241.g8f8d3829.el9.x86 30/184 2026-02-13T02:19:13.122 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 31/184 2026-02-13T02:19:13.147 INFO:teuthology.orchestra.run.trial028.stdout: Installing : zip-3.0-35.el9.x86_64 32/184 2026-02-13T02:19:13.153 INFO:teuthology.orchestra.run.trial028.stdout: Installing : luarocks-3.9.2-5.el9.noarch 33/184 2026-02-13T02:19:13.155 INFO:teuthology.orchestra.run.trial028.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 34/184 2026-02-13T02:19:13.158 INFO:teuthology.orchestra.run.trial028.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/184 2026-02-13T02:19:13.161 INFO:teuthology.orchestra.run.trial028.stdout: Installing : perl-threads-shared-1.61-460.el9.x86_64 36/184 2026-02-13T02:19:13.188 INFO:teuthology.orchestra.run.trial028.stdout: Installing : perl-Thread-Queue-3.14-460.el9.noarch 37/184 2026-02-13T02:19:13.239 INFO:teuthology.orchestra.run.trial028.stdout: Installing : automake-1.16.2-8.el9.noarch 38/184 2026-02-13T02:19:13.245 INFO:teuthology.orchestra.run.trial027.stdout: Upgrading : openssl-fips-provider-1:3.5.5-1.el9.x86_64 44/184 2026-02-13T02:19:13.248 INFO:teuthology.orchestra.run.trial027.stdout: Upgrading : librados2-2:20.3.0-5241.g8f8d3829.el9.x86_64 45/184 2026-02-13T02:19:13.252 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/184 2026-02-13T02:19:13.258 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/184 2026-02-13T02:19:13.267 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 41/184 2026-02-13T02:19:13.284 INFO:teuthology.orchestra.run.trial028.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 42/184 2026-02-13T02:19:13.291 INFO:teuthology.orchestra.run.trial028.stdout: Upgrading : openssl-libs-1:3.5.5-1.el9.x86_64 43/184 2026-02-13T02:19:13.378 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: librados2-2:20.3.0-5241.g8f8d3829.el9.x86_64 45/184 2026-02-13T02:19:13.379 INFO:teuthology.orchestra.run.trial027.stdout: Upgrading : librbd1-2:20.3.0-5241.g8f8d3829.el9.x86_64 46/184 2026-02-13T02:19:13.428 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: librbd1-2:20.3.0-5241.g8f8d3829.el9.x86_64 46/184 2026-02-13T02:19:13.429 INFO:teuthology.orchestra.run.trial027.stdout: Installing : libcephfs2-2:20.3.0-5241.g8f8d3829.el9.x86_64 47/184 2026-02-13T02:19:13.445 INFO:teuthology.orchestra.run.trial028.stdout: Upgrading : openssl-fips-provider-1:3.5.5-1.el9.x86_64 44/184 2026-02-13T02:19:13.449 INFO:teuthology.orchestra.run.trial028.stdout: Upgrading : librados2-2:20.3.0-5241.g8f8d3829.el9.x86_64 45/184 2026-02-13T02:19:13.458 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: libcephfs2-2:20.3.0-5241.g8f8d3829.el9.x86_64 47/184 2026-02-13T02:19:13.461 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-rados-2:20.3.0-5241.g8f8d3829.el9.x86_64 48/184 2026-02-13T02:19:13.469 INFO:teuthology.orchestra.run.trial027.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 49/184 2026-02-13T02:19:13.471 INFO:teuthology.orchestra.run.trial027.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 50/184 2026-02-13T02:19:13.531 INFO:teuthology.orchestra.run.trial027.stdout: Installing : postgresql-private-libs-13.23-1.el9.x86_64 51/184 2026-02-13T02:19:13.544 INFO:teuthology.orchestra.run.trial027.stdout: Installing : postgresql-13.23-1.el9.x86_64 52/184 2026-02-13T02:19:13.553 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-cephfs-2:20.3.0-5241.g8f8d3829.el9.x86_6 53/184 2026-02-13T02:19:13.554 INFO:teuthology.orchestra.run.trial027.stdout: Installing : libcephsqlite-2:20.3.0-5241.g8f8d3829.el9.x86_64 54/184 2026-02-13T02:19:13.576 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5241.g8f8d3829.el9.x86_64 54/184 2026-02-13T02:19:13.577 INFO:teuthology.orchestra.run.trial027.stdout: Installing : libradosstriper1-2:20.3.0-5241.g8f8d3829.el9.x86 55/184 2026-02-13T02:19:13.584 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: librados2-2:20.3.0-5241.g8f8d3829.el9.x86_64 45/184 2026-02-13T02:19:13.586 INFO:teuthology.orchestra.run.trial028.stdout: Upgrading : librbd1-2:20.3.0-5241.g8f8d3829.el9.x86_64 46/184 2026-02-13T02:19:13.604 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5241.g8f8d3829.el9.x86 55/184 2026-02-13T02:19:13.611 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-rbd-2:20.3.0-5241.g8f8d3829.el9.x86_64 56/184 2026-02-13T02:19:13.619 INFO:teuthology.orchestra.run.trial027.stdout: Installing : librados-devel-2:20.3.0-5241.g8f8d3829.el9.x86_6 57/184 2026-02-13T02:19:13.632 INFO:teuthology.orchestra.run.trial027.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 58/184 2026-02-13T02:19:13.635 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: librbd1-2:20.3.0-5241.g8f8d3829.el9.x86_64 46/184 2026-02-13T02:19:13.637 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libcephfs2-2:20.3.0-5241.g8f8d3829.el9.x86_64 47/184 2026-02-13T02:19:13.666 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: libcephfs2-2:20.3.0-5241.g8f8d3829.el9.x86_64 47/184 2026-02-13T02:19:13.668 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-rados-2:20.3.0-5241.g8f8d3829.el9.x86_64 48/184 2026-02-13T02:19:13.676 INFO:teuthology.orchestra.run.trial028.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 49/184 2026-02-13T02:19:13.679 INFO:teuthology.orchestra.run.trial028.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 50/184 2026-02-13T02:19:13.688 INFO:teuthology.orchestra.run.trial027.stdout: Installing : thrift-0.15.0-4.el9.x86_64 59/184 2026-02-13T02:19:13.717 INFO:teuthology.orchestra.run.trial027.stdout: Upgrading : openssl-1:3.5.5-1.el9.x86_64 60/184 2026-02-13T02:19:13.723 INFO:teuthology.orchestra.run.trial027.stdout: Upgrading : iproute-6.17.0-2.el9.x86_64 61/184 2026-02-13T02:19:13.727 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 62/184 2026-02-13T02:19:13.740 INFO:teuthology.orchestra.run.trial028.stdout: Installing : postgresql-private-libs-13.23-1.el9.x86_64 51/184 2026-02-13T02:19:13.741 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 63/184 2026-02-13T02:19:13.745 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 64/184 2026-02-13T02:19:13.750 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 65/184 2026-02-13T02:19:13.755 INFO:teuthology.orchestra.run.trial028.stdout: Installing : postgresql-13.23-1.el9.x86_64 52/184 2026-02-13T02:19:13.761 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 66/184 2026-02-13T02:19:13.762 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-cephfs-2:20.3.0-5241.g8f8d3829.el9.x86_6 53/184 2026-02-13T02:19:13.763 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libcephsqlite-2:20.3.0-5241.g8f8d3829.el9.x86_64 54/184 2026-02-13T02:19:13.766 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 67/184 2026-02-13T02:19:13.775 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 68/184 2026-02-13T02:19:13.779 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 69/184 2026-02-13T02:19:13.786 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5241.g8f8d3829.el9.x86_64 54/184 2026-02-13T02:19:13.787 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 70/184 2026-02-13T02:19:13.787 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libradosstriper1-2:20.3.0-5241.g8f8d3829.el9.x86 55/184 2026-02-13T02:19:13.815 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5241.g8f8d3829.el9.x86 55/184 2026-02-13T02:19:13.823 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-rbd-2:20.3.0-5241.g8f8d3829.el9.x86_64 56/184 2026-02-13T02:19:13.827 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 71/184 2026-02-13T02:19:13.831 INFO:teuthology.orchestra.run.trial028.stdout: Installing : librados-devel-2:20.3.0-5241.g8f8d3829.el9.x86_6 57/184 2026-02-13T02:19:13.843 INFO:teuthology.orchestra.run.trial028.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 58/184 2026-02-13T02:19:13.900 INFO:teuthology.orchestra.run.trial028.stdout: Installing : thrift-0.15.0-4.el9.x86_64 59/184 2026-02-13T02:19:13.930 INFO:teuthology.orchestra.run.trial028.stdout: Upgrading : openssl-1:3.5.5-1.el9.x86_64 60/184 2026-02-13T02:19:13.936 INFO:teuthology.orchestra.run.trial028.stdout: Upgrading : iproute-6.17.0-2.el9.x86_64 61/184 2026-02-13T02:19:13.939 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 62/184 2026-02-13T02:19:13.953 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 63/184 2026-02-13T02:19:13.958 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 64/184 2026-02-13T02:19:13.963 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 65/184 2026-02-13T02:19:13.974 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 66/184 2026-02-13T02:19:13.979 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 67/184 2026-02-13T02:19:13.988 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 68/184 2026-02-13T02:19:13.992 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 69/184 2026-02-13T02:19:14.000 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 70/184 2026-02-13T02:19:14.039 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 71/184 2026-02-13T02:19:14.148 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 72/184 2026-02-13T02:19:14.158 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 73/184 2026-02-13T02:19:14.162 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 74/184 2026-02-13T02:19:14.168 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 75/184 2026-02-13T02:19:14.171 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 76/184 2026-02-13T02:19:14.178 INFO:teuthology.orchestra.run.trial027.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 77/184 2026-02-13T02:19:14.180 INFO:teuthology.orchestra.run.trial027.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 78/184 2026-02-13T02:19:14.181 INFO:teuthology.orchestra.run.trial027.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 79/184 2026-02-13T02:19:14.209 INFO:teuthology.orchestra.run.trial027.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 80/184 2026-02-13T02:19:14.252 INFO:teuthology.orchestra.run.trial027.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 81/184 2026-02-13T02:19:14.256 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 82/184 2026-02-13T02:19:14.259 INFO:teuthology.orchestra.run.trial027.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 83/184 2026-02-13T02:19:14.266 INFO:teuthology.orchestra.run.trial027.stdout: Installing : uuid-1.6.2-55.el9.x86_64 84/184 2026-02-13T02:19:14.293 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 85/184 2026-02-13T02:19:14.299 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/184 2026-02-13T02:19:14.303 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 87/184 2026-02-13T02:19:14.309 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 88/184 2026-02-13T02:19:14.312 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 89/184 2026-02-13T02:19:14.319 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 90/184 2026-02-13T02:19:14.323 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 91/184 2026-02-13T02:19:14.351 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 92/184 2026-02-13T02:19:14.360 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 93/184 2026-02-13T02:19:14.362 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 72/184 2026-02-13T02:19:14.366 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 94/184 2026-02-13T02:19:14.372 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 73/184 2026-02-13T02:19:14.373 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 95/184 2026-02-13T02:19:14.376 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 74/184 2026-02-13T02:19:14.382 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 75/184 2026-02-13T02:19:14.385 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 76/184 2026-02-13T02:19:14.391 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 77/184 2026-02-13T02:19:14.393 INFO:teuthology.orchestra.run.trial028.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 78/184 2026-02-13T02:19:14.395 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 79/184 2026-02-13T02:19:14.407 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 96/184 2026-02-13T02:19:14.418 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-devel-3.9.25-2.el9.x86_64 97/184 2026-02-13T02:19:14.423 INFO:teuthology.orchestra.run.trial028.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 80/184 2026-02-13T02:19:14.466 INFO:teuthology.orchestra.run.trial028.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 81/184 2026-02-13T02:19:14.469 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 82/184 2026-02-13T02:19:14.473 INFO:teuthology.orchestra.run.trial028.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 83/184 2026-02-13T02:19:14.479 INFO:teuthology.orchestra.run.trial028.stdout: Installing : uuid-1.6.2-55.el9.x86_64 84/184 2026-02-13T02:19:14.507 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 85/184 2026-02-13T02:19:14.513 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/184 2026-02-13T02:19:14.517 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 87/184 2026-02-13T02:19:14.523 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 88/184 2026-02-13T02:19:14.526 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 89/184 2026-02-13T02:19:14.533 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 90/184 2026-02-13T02:19:14.537 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 91/184 2026-02-13T02:19:14.564 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 92/184 2026-02-13T02:19:14.574 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 93/184 2026-02-13T02:19:14.580 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 94/184 2026-02-13T02:19:14.587 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 95/184 2026-02-13T02:19:14.621 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 96/184 2026-02-13T02:19:14.632 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-devel-3.9.25-2.el9.x86_64 97/184 2026-02-13T02:19:14.639 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 98/184 2026-02-13T02:19:14.662 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 99/184 2026-02-13T02:19:14.666 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 100/184 2026-02-13T02:19:14.724 INFO:teuthology.orchestra.run.trial027.stdout: Installing : openblas-0.3.29-1.el9.x86_64 101/184 2026-02-13T02:19:14.726 INFO:teuthology.orchestra.run.trial027.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 102/184 2026-02-13T02:19:14.746 INFO:teuthology.orchestra.run.trial027.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 103/184 2026-02-13T02:19:14.854 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 98/184 2026-02-13T02:19:14.878 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 99/184 2026-02-13T02:19:14.882 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 100/184 2026-02-13T02:19:14.940 INFO:teuthology.orchestra.run.trial028.stdout: Installing : openblas-0.3.29-1.el9.x86_64 101/184 2026-02-13T02:19:14.942 INFO:teuthology.orchestra.run.trial028.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 102/184 2026-02-13T02:19:14.963 INFO:teuthology.orchestra.run.trial028.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 103/184 2026-02-13T02:19:15.070 INFO:teuthology.orchestra.run.trial027.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 104/184 2026-02-13T02:19:15.143 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 105/184 2026-02-13T02:19:15.287 INFO:teuthology.orchestra.run.trial028.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 104/184 2026-02-13T02:19:15.361 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 105/184 2026-02-13T02:19:15.392 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5241.g8f8d3829.el9.x86_64 129/184 2026-02-13T02:19:15.392 INFO:teuthology.orchestra.run.trial107.stdout:skipping the directory /sys 2026-02-13T02:19:15.392 INFO:teuthology.orchestra.run.trial107.stdout:skipping the directory /proc 2026-02-13T02:19:15.392 INFO:teuthology.orchestra.run.trial107.stdout:skipping the directory /mnt 2026-02-13T02:19:15.392 INFO:teuthology.orchestra.run.trial107.stdout:skipping the directory /var/tmp 2026-02-13T02:19:15.393 INFO:teuthology.orchestra.run.trial107.stdout:skipping the directory /home 2026-02-13T02:19:15.393 INFO:teuthology.orchestra.run.trial107.stdout:skipping the directory /root 2026-02-13T02:19:15.393 INFO:teuthology.orchestra.run.trial107.stdout:skipping the directory /tmp 2026-02-13T02:19:15.393 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:19:15.496 INFO:teuthology.orchestra.run.trial107.stdout: Installing : ceph-mds-2:20.3.0-5241.g8f8d3829.el9.x86_64 130/184 2026-02-13T02:19:15.520 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: ceph-mds-2:20.3.0-5241.g8f8d3829.el9.x86_64 130/184 2026-02-13T02:19:15.520 INFO:teuthology.orchestra.run.trial107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T02:19:15.520 INFO:teuthology.orchestra.run.trial107.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-13T02:19:15.520 INFO:teuthology.orchestra.run.trial107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-13T02:19:15.520 INFO:teuthology.orchestra.run.trial107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-13T02:19:15.520 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:19:15.741 INFO:teuthology.orchestra.run.trial107.stdout: Installing : ceph-mon-2:20.3.0-5241.g8f8d3829.el9.x86_64 131/184 2026-02-13T02:19:15.763 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: ceph-mon-2:20.3.0-5241.g8f8d3829.el9.x86_64 131/184 2026-02-13T02:19:15.763 INFO:teuthology.orchestra.run.trial107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T02:19:15.763 INFO:teuthology.orchestra.run.trial107.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-13T02:19:15.763 INFO:teuthology.orchestra.run.trial107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-13T02:19:15.763 INFO:teuthology.orchestra.run.trial107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-13T02:19:15.763 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:19:15.769 INFO:teuthology.orchestra.run.trial107.stdout: Installing : mailcap-2.1.49-5.el9.noarch 132/184 2026-02-13T02:19:15.770 INFO:teuthology.orchestra.run.trial107.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 133/184 2026-02-13T02:19:15.788 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 134/184 2026-02-13T02:19:15.789 INFO:teuthology.orchestra.run.trial107.stdout:Creating group 'qat' with GID 992. 2026-02-13T02:19:15.789 INFO:teuthology.orchestra.run.trial107.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-13T02:19:15.789 INFO:teuthology.orchestra.run.trial107.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-13T02:19:15.789 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:19:15.797 INFO:teuthology.orchestra.run.trial107.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 134/184 2026-02-13T02:19:15.807 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 106/184 2026-02-13T02:19:15.821 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 134/184 2026-02-13T02:19:15.821 INFO:teuthology.orchestra.run.trial107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-13T02:19:15.821 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:19:15.822 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 107/184 2026-02-13T02:19:15.825 INFO:teuthology.orchestra.run.trial027.stdout: Installing : libzstd-devel-1.5.5-1.el9.x86_64 108/184 2026-02-13T02:19:15.827 INFO:teuthology.orchestra.run.trial027.stdout: Installing : libuuid-devel-2.37.4-21.el9.x86_64 109/184 2026-02-13T02:19:15.833 INFO:teuthology.orchestra.run.trial027.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 110/184 2026-02-13T02:19:15.857 INFO:teuthology.orchestra.run.trial107.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 135/184 2026-02-13T02:19:16.029 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 106/184 2026-02-13T02:19:16.044 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 107/184 2026-02-13T02:19:16.047 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libzstd-devel-1.5.5-1.el9.x86_64 108/184 2026-02-13T02:19:16.049 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libuuid-devel-2.37.4-21.el9.x86_64 109/184 2026-02-13T02:19:16.055 INFO:teuthology.orchestra.run.trial028.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 110/184 2026-02-13T02:19:16.114 INFO:teuthology.orchestra.run.trial027.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 111/184 2026-02-13T02:19:16.116 INFO:teuthology.orchestra.run.trial027.stdout: Installing : librgw2-2:20.3.0-5241.g8f8d3829.el9.x86_64 112/184 2026-02-13T02:19:16.134 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: librgw2-2:20.3.0-5241.g8f8d3829.el9.x86_64 112/184 2026-02-13T02:19:16.135 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-rgw-2:20.3.0-5241.g8f8d3829.el9.x86_64 113/184 2026-02-13T02:19:16.342 INFO:teuthology.orchestra.run.trial028.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 111/184 2026-02-13T02:19:16.343 INFO:teuthology.orchestra.run.trial028.stdout: Installing : librgw2-2:20.3.0-5241.g8f8d3829.el9.x86_64 112/184 2026-02-13T02:19:16.364 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: librgw2-2:20.3.0-5241.g8f8d3829.el9.x86_64 112/184 2026-02-13T02:19:16.365 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-rgw-2:20.3.0-5241.g8f8d3829.el9.x86_64 113/184 2026-02-13T02:19:16.491 INFO:teuthology.orchestra.run.trial107.stdout: Installing : cryptsetup-2.8.1-2.el9.x86_64 136/184 2026-02-13T02:19:17.220 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: ceph-common-2:20.3.0-5241.g8f8d3829.el9.x86_64 114/184 2026-02-13T02:19:17.222 INFO:teuthology.orchestra.run.trial027.stdout: Installing : ceph-common-2:20.3.0-5241.g8f8d3829.el9.x86_64 114/184 2026-02-13T02:19:17.242 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: ceph-common-2:20.3.0-5241.g8f8d3829.el9.x86_64 114/184 2026-02-13T02:19:17.257 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/184 2026-02-13T02:19:17.264 INFO:teuthology.orchestra.run.trial107.stdout: Installing : ceph-osd-classic-2:20.3.0-5241.g8f8d3829.el9.x86 137/184 2026-02-13T02:19:17.274 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/184 2026-02-13T02:19:17.326 INFO:teuthology.orchestra.run.trial107.stdout: Installing : ceph-osd-crimson-2:20.3.0-5241.g8f8d3829.el9.x86 138/184 2026-02-13T02:19:17.328 INFO:teuthology.orchestra.run.trial107.stdout: Installing : ceph-volume-2:20.3.0-5241.g8f8d3829.el9.noarch 139/184 2026-02-13T02:19:17.342 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: ceph-volume-2:20.3.0-5241.g8f8d3829.el9.noarch 139/184 2026-02-13T02:19:17.342 INFO:teuthology.orchestra.run.trial107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T02:19:17.342 INFO:teuthology.orchestra.run.trial107.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-13T02:19:17.342 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:19:17.350 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/184 2026-02-13T02:19:17.352 INFO:teuthology.orchestra.run.trial107.stdout: Installing : ceph-osd-2:20.3.0-5241.g8f8d3829.el9.x86_64 140/184 2026-02-13T02:19:17.359 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 118/184 2026-02-13T02:19:17.375 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: ceph-osd-2:20.3.0-5241.g8f8d3829.el9.x86_64 140/184 2026-02-13T02:19:17.375 INFO:teuthology.orchestra.run.trial107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T02:19:17.376 INFO:teuthology.orchestra.run.trial107.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-13T02:19:17.376 INFO:teuthology.orchestra.run.trial107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-13T02:19:17.376 INFO:teuthology.orchestra.run.trial107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-13T02:19:17.376 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:19:17.382 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/184 2026-02-13T02:19:17.412 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 120/184 2026-02-13T02:19:17.436 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: cephadm-2:20.3.0-5241.g8f8d3829.el9.noarch 141/184 2026-02-13T02:19:17.438 INFO:teuthology.orchestra.run.trial107.stdout: Installing : cephadm-2:20.3.0-5241.g8f8d3829.el9.noarch 141/184 2026-02-13T02:19:17.447 INFO:teuthology.orchestra.run.trial107.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5241.g8f8d3829.e 142/184 2026-02-13T02:19:17.463 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/184 2026-02-13T02:19:17.469 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/184 2026-02-13T02:19:17.470 INFO:teuthology.orchestra.run.trial107.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5241.g8f8d3829. 143/184 2026-02-13T02:19:17.472 INFO:teuthology.orchestra.run.trial107.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5241.g8f8d3829.el9.noa 144/184 2026-02-13T02:19:17.472 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/184 2026-02-13T02:19:17.474 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-common-2:20.3.0-5241.g8f8d3829.el9.x86_64 114/184 2026-02-13T02:19:17.477 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-common-2:20.3.0-5241.g8f8d3829.el9.x86_64 114/184 2026-02-13T02:19:17.478 INFO:teuthology.orchestra.run.trial027.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 124/184 2026-02-13T02:19:17.480 INFO:teuthology.orchestra.run.trial027.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 125/184 2026-02-13T02:19:17.481 INFO:teuthology.orchestra.run.trial027.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 126/184 2026-02-13T02:19:17.496 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-common-2:20.3.0-5241.g8f8d3829.el9.x86_64 114/184 2026-02-13T02:19:17.498 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 126/184 2026-02-13T02:19:17.509 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/184 2026-02-13T02:19:17.525 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/184 2026-02-13T02:19:17.602 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/184 2026-02-13T02:19:17.612 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 118/184 2026-02-13T02:19:17.636 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/184 2026-02-13T02:19:17.666 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 120/184 2026-02-13T02:19:17.716 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/184 2026-02-13T02:19:17.722 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/184 2026-02-13T02:19:17.726 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/184 2026-02-13T02:19:17.730 INFO:teuthology.orchestra.run.trial028.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 124/184 2026-02-13T02:19:17.732 INFO:teuthology.orchestra.run.trial028.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 125/184 2026-02-13T02:19:17.733 INFO:teuthology.orchestra.run.trial028.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 126/184 2026-02-13T02:19:17.746 INFO:teuthology.orchestra.run.trial027.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 127/184 2026-02-13T02:19:17.750 INFO:teuthology.orchestra.run.trial027.stdout: Installing : ceph-base-2:20.3.0-5241.g8f8d3829.el9.x86_64 128/184 2026-02-13T02:19:17.751 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 126/184 2026-02-13T02:19:17.785 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: ceph-base-2:20.3.0-5241.g8f8d3829.el9.x86_64 128/184 2026-02-13T02:19:17.786 INFO:teuthology.orchestra.run.trial027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-13T02:19:17.786 INFO:teuthology.orchestra.run.trial027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-13T02:19:17.786 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:19:17.789 INFO:teuthology.orchestra.run.trial027.stdout: Installing : ceph-selinux-2:20.3.0-5241.g8f8d3829.el9.x86_64 129/184 2026-02-13T02:19:18.001 INFO:teuthology.orchestra.run.trial028.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 127/184 2026-02-13T02:19:18.004 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-base-2:20.3.0-5241.g8f8d3829.el9.x86_64 128/184 2026-02-13T02:19:18.045 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-base-2:20.3.0-5241.g8f8d3829.el9.x86_64 128/184 2026-02-13T02:19:18.045 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-13T02:19:18.045 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-13T02:19:18.045 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:19:18.049 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-selinux-2:20.3.0-5241.g8f8d3829.el9.x86_64 129/184 2026-02-13T02:19:19.039 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5241.g8f8d3829.el9.noa 144/184 2026-02-13T02:19:19.046 INFO:teuthology.orchestra.run.trial107.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5241.g8f8d3829.el9.n 145/184 2026-02-13T02:19:19.509 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5241.g8f8d3829.el9.n 145/184 2026-02-13T02:19:19.509 INFO:teuthology.orchestra.run.trial107.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5241.g8f8 146/184 2026-02-13T02:19:19.571 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5241.g8f8 146/184 2026-02-13T02:19:19.613 INFO:teuthology.orchestra.run.trial107.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5241.g8f8d3829.el 147/184 2026-02-13T02:19:19.615 INFO:teuthology.orchestra.run.trial107.stdout: Installing : ceph-mgr-2:20.3.0-5241.g8f8d3829.el9.x86_64 148/184 2026-02-13T02:19:19.635 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5241.g8f8d3829.el9.x86_64 148/184 2026-02-13T02:19:19.635 INFO:teuthology.orchestra.run.trial107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T02:19:19.635 INFO:teuthology.orchestra.run.trial107.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-13T02:19:19.635 INFO:teuthology.orchestra.run.trial107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-13T02:19:19.636 INFO:teuthology.orchestra.run.trial107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-13T02:19:19.636 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:19:19.646 INFO:teuthology.orchestra.run.trial107.stdout: Installing : ceph-mgr-rook-2:20.3.0-5241.g8f8d3829.el9.noarch 149/184 2026-02-13T02:19:19.656 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5241.g8f8d3829.el9.noarch 149/184 2026-02-13T02:19:20.240 INFO:teuthology.orchestra.run.trial107.stdout: Installing : ceph-2:20.3.0-5241.g8f8d3829.el9.x86_64 150/184 2026-02-13T02:19:20.243 INFO:teuthology.orchestra.run.trial107.stdout: Installing : ceph-radosgw-2:20.3.0-5241.g8f8d3829.el9.x86_64 151/184 2026-02-13T02:19:20.265 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5241.g8f8d3829.el9.x86_64 151/184 2026-02-13T02:19:20.265 INFO:teuthology.orchestra.run.trial107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T02:19:20.265 INFO:teuthology.orchestra.run.trial107.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-13T02:19:20.265 INFO:teuthology.orchestra.run.trial107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-13T02:19:20.266 INFO:teuthology.orchestra.run.trial107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-13T02:19:20.266 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:19:20.274 INFO:teuthology.orchestra.run.trial107.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5241.g8f8d3 152/184 2026-02-13T02:19:20.295 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5241.g8f8d3 152/184 2026-02-13T02:19:20.295 INFO:teuthology.orchestra.run.trial107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T02:19:20.295 INFO:teuthology.orchestra.run.trial107.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-13T02:19:20.295 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:19:20.309 INFO:teuthology.orchestra.run.trial107.stdout: Installing : cephfs-mirror-2:20.3.0-5241.g8f8d3829.el9.x86_64 153/184 2026-02-13T02:19:20.329 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: cephfs-mirror-2:20.3.0-5241.g8f8d3829.el9.x86_64 153/184 2026-02-13T02:19:20.329 INFO:teuthology.orchestra.run.trial107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T02:19:20.330 INFO:teuthology.orchestra.run.trial107.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2026-02-13T02:19:20.330 INFO:teuthology.orchestra.run.trial107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2026-02-13T02:19:20.330 INFO:teuthology.orchestra.run.trial107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2026-02-13T02:19:20.330 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:19:20.456 INFO:teuthology.orchestra.run.trial107.stdout: Installing : rbd-mirror-2:20.3.0-5241.g8f8d3829.el9.x86_64 154/184 2026-02-13T02:19:20.477 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5241.g8f8d3829.el9.x86_64 154/184 2026-02-13T02:19:20.477 INFO:teuthology.orchestra.run.trial107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T02:19:20.477 INFO:teuthology.orchestra.run.trial107.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-13T02:19:20.477 INFO:teuthology.orchestra.run.trial107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-13T02:19:20.477 INFO:teuthology.orchestra.run.trial107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-13T02:19:20.477 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:19:22.478 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5241.g8f8d3829.el9.x86_64 129/184 2026-02-13T02:19:22.479 INFO:teuthology.orchestra.run.trial027.stdout:skipping the directory /sys 2026-02-13T02:19:22.479 INFO:teuthology.orchestra.run.trial027.stdout:skipping the directory /proc 2026-02-13T02:19:22.479 INFO:teuthology.orchestra.run.trial027.stdout:skipping the directory /mnt 2026-02-13T02:19:22.479 INFO:teuthology.orchestra.run.trial027.stdout:skipping the directory /var/tmp 2026-02-13T02:19:22.479 INFO:teuthology.orchestra.run.trial027.stdout:skipping the directory /home 2026-02-13T02:19:22.479 INFO:teuthology.orchestra.run.trial027.stdout:skipping the directory /root 2026-02-13T02:19:22.479 INFO:teuthology.orchestra.run.trial027.stdout:skipping the directory /tmp 2026-02-13T02:19:22.479 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:19:22.581 INFO:teuthology.orchestra.run.trial027.stdout: Installing : ceph-mds-2:20.3.0-5241.g8f8d3829.el9.x86_64 130/184 2026-02-13T02:19:22.604 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: ceph-mds-2:20.3.0-5241.g8f8d3829.el9.x86_64 130/184 2026-02-13T02:19:22.604 INFO:teuthology.orchestra.run.trial027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T02:19:22.604 INFO:teuthology.orchestra.run.trial027.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-13T02:19:22.604 INFO:teuthology.orchestra.run.trial027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-13T02:19:22.604 INFO:teuthology.orchestra.run.trial027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-13T02:19:22.605 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:19:22.796 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5241.g8f8d3829.el9.x86_64 129/184 2026-02-13T02:19:22.796 INFO:teuthology.orchestra.run.trial028.stdout:skipping the directory /sys 2026-02-13T02:19:22.797 INFO:teuthology.orchestra.run.trial028.stdout:skipping the directory /proc 2026-02-13T02:19:22.797 INFO:teuthology.orchestra.run.trial028.stdout:skipping the directory /mnt 2026-02-13T02:19:22.797 INFO:teuthology.orchestra.run.trial028.stdout:skipping the directory /var/tmp 2026-02-13T02:19:22.797 INFO:teuthology.orchestra.run.trial028.stdout:skipping the directory /home 2026-02-13T02:19:22.797 INFO:teuthology.orchestra.run.trial028.stdout:skipping the directory /root 2026-02-13T02:19:22.797 INFO:teuthology.orchestra.run.trial028.stdout:skipping the directory /tmp 2026-02-13T02:19:22.797 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:19:22.818 INFO:teuthology.orchestra.run.trial027.stdout: Installing : ceph-mon-2:20.3.0-5241.g8f8d3829.el9.x86_64 131/184 2026-02-13T02:19:22.840 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: ceph-mon-2:20.3.0-5241.g8f8d3829.el9.x86_64 131/184 2026-02-13T02:19:22.840 INFO:teuthology.orchestra.run.trial027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T02:19:22.840 INFO:teuthology.orchestra.run.trial027.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-13T02:19:22.840 INFO:teuthology.orchestra.run.trial027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-13T02:19:22.840 INFO:teuthology.orchestra.run.trial027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-13T02:19:22.840 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:19:22.846 INFO:teuthology.orchestra.run.trial027.stdout: Installing : mailcap-2.1.49-5.el9.noarch 132/184 2026-02-13T02:19:22.848 INFO:teuthology.orchestra.run.trial027.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 133/184 2026-02-13T02:19:22.865 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 134/184 2026-02-13T02:19:22.866 INFO:teuthology.orchestra.run.trial027.stdout:Creating group 'qat' with GID 992. 2026-02-13T02:19:22.866 INFO:teuthology.orchestra.run.trial027.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-13T02:19:22.866 INFO:teuthology.orchestra.run.trial027.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-13T02:19:22.866 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:19:22.874 INFO:teuthology.orchestra.run.trial027.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 134/184 2026-02-13T02:19:22.897 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 134/184 2026-02-13T02:19:22.897 INFO:teuthology.orchestra.run.trial027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-13T02:19:22.897 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:19:22.900 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-mds-2:20.3.0-5241.g8f8d3829.el9.x86_64 130/184 2026-02-13T02:19:22.923 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-mds-2:20.3.0-5241.g8f8d3829.el9.x86_64 130/184 2026-02-13T02:19:22.923 INFO:teuthology.orchestra.run.trial028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T02:19:22.924 INFO:teuthology.orchestra.run.trial028.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-13T02:19:22.924 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-13T02:19:22.924 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-13T02:19:22.924 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:19:22.934 INFO:teuthology.orchestra.run.trial027.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 135/184 2026-02-13T02:19:23.140 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-mon-2:20.3.0-5241.g8f8d3829.el9.x86_64 131/184 2026-02-13T02:19:23.162 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-mon-2:20.3.0-5241.g8f8d3829.el9.x86_64 131/184 2026-02-13T02:19:23.162 INFO:teuthology.orchestra.run.trial028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T02:19:23.162 INFO:teuthology.orchestra.run.trial028.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-13T02:19:23.162 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-13T02:19:23.162 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-13T02:19:23.162 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:19:23.168 INFO:teuthology.orchestra.run.trial028.stdout: Installing : mailcap-2.1.49-5.el9.noarch 132/184 2026-02-13T02:19:23.170 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 133/184 2026-02-13T02:19:23.188 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 134/184 2026-02-13T02:19:23.188 INFO:teuthology.orchestra.run.trial028.stdout:Creating group 'qat' with GID 992. 2026-02-13T02:19:23.188 INFO:teuthology.orchestra.run.trial028.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-13T02:19:23.188 INFO:teuthology.orchestra.run.trial028.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-13T02:19:23.189 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:19:23.197 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 134/184 2026-02-13T02:19:23.222 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 134/184 2026-02-13T02:19:23.222 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-13T02:19:23.222 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:19:23.256 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 135/184 2026-02-13T02:19:23.552 INFO:teuthology.orchestra.run.trial027.stdout: Installing : cryptsetup-2.8.1-2.el9.x86_64 136/184 2026-02-13T02:19:23.887 INFO:teuthology.orchestra.run.trial028.stdout: Installing : cryptsetup-2.8.1-2.el9.x86_64 136/184 2026-02-13T02:19:23.978 INFO:teuthology.orchestra.run.trial107.stdout: Installing : ceph-test-2:20.3.0-5241.g8f8d3829.el9.x86_64 155/184 2026-02-13T02:19:23.992 INFO:teuthology.orchestra.run.trial107.stdout: Installing : xfsprogs-devel-6.4.0-7.el9.x86_64 156/184 2026-02-13T02:19:23.995 INFO:teuthology.orchestra.run.trial107.stdout: Installing : elfutils-libelf-devel-0.194-1.el9.x86_64 157/184 2026-02-13T02:19:24.068 INFO:teuthology.orchestra.run.trial107.stdout: Installing : cephfs-shell-2:20.3.0-5241.g8f8d3829.el9.x86_64 158/184 2026-02-13T02:19:24.086 INFO:teuthology.orchestra.run.trial107.stdout: Installing : postgresql-contrib-13.23-1.el9.x86_64 159/184 2026-02-13T02:19:24.090 INFO:teuthology.orchestra.run.trial107.stdout: Upgrading : iproute-tc-6.17.0-2.el9.x86_64 160/184 2026-02-13T02:19:24.092 INFO:teuthology.orchestra.run.trial107.stdout: Installing : libcephfs-devel-2:20.3.0-5241.g8f8d3829.el9.x86_ 161/184 2026-02-13T02:19:24.211 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: postgresql-server-13.23-1.el9.x86_64 162/184 2026-02-13T02:19:24.215 INFO:teuthology.orchestra.run.trial107.stdout: Installing : postgresql-server-13.23-1.el9.x86_64 162/184 2026-02-13T02:19:24.232 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: postgresql-server-13.23-1.el9.x86_64 162/184 2026-02-13T02:19:24.238 INFO:teuthology.orchestra.run.trial107.stdout: Installing : cephfs-top-2:20.3.0-5241.g8f8d3829.el9.noarch 163/184 2026-02-13T02:19:24.247 INFO:teuthology.orchestra.run.trial107.stdout: Installing : rbd-fuse-2:20.3.0-5241.g8f8d3829.el9.x86_64 164/184 2026-02-13T02:19:24.290 INFO:teuthology.orchestra.run.trial107.stdout: Installing : rbd-nbd-2:20.3.0-5241.g8f8d3829.el9.x86_64 165/184 2026-02-13T02:19:24.309 INFO:teuthology.orchestra.run.trial027.stdout: Installing : ceph-osd-classic-2:20.3.0-5241.g8f8d3829.el9.x86 137/184 2026-02-13T02:19:24.371 INFO:teuthology.orchestra.run.trial027.stdout: Installing : ceph-osd-crimson-2:20.3.0-5241.g8f8d3829.el9.x86 138/184 2026-02-13T02:19:24.374 INFO:teuthology.orchestra.run.trial027.stdout: Installing : ceph-volume-2:20.3.0-5241.g8f8d3829.el9.noarch 139/184 2026-02-13T02:19:24.388 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: ceph-volume-2:20.3.0-5241.g8f8d3829.el9.noarch 139/184 2026-02-13T02:19:24.388 INFO:teuthology.orchestra.run.trial027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T02:19:24.388 INFO:teuthology.orchestra.run.trial027.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-13T02:19:24.388 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:19:24.399 INFO:teuthology.orchestra.run.trial027.stdout: Installing : ceph-osd-2:20.3.0-5241.g8f8d3829.el9.x86_64 140/184 2026-02-13T02:19:24.424 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: ceph-osd-2:20.3.0-5241.g8f8d3829.el9.x86_64 140/184 2026-02-13T02:19:24.424 INFO:teuthology.orchestra.run.trial027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T02:19:24.424 INFO:teuthology.orchestra.run.trial027.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-13T02:19:24.424 INFO:teuthology.orchestra.run.trial027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-13T02:19:24.424 INFO:teuthology.orchestra.run.trial027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-13T02:19:24.424 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:19:24.484 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: cephadm-2:20.3.0-5241.g8f8d3829.el9.noarch 141/184 2026-02-13T02:19:24.486 INFO:teuthology.orchestra.run.trial027.stdout: Installing : cephadm-2:20.3.0-5241.g8f8d3829.el9.noarch 141/184 2026-02-13T02:19:24.495 INFO:teuthology.orchestra.run.trial027.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5241.g8f8d3829.e 142/184 2026-02-13T02:19:24.519 INFO:teuthology.orchestra.run.trial027.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5241.g8f8d3829. 143/184 2026-02-13T02:19:24.521 INFO:teuthology.orchestra.run.trial027.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5241.g8f8d3829.el9.noa 144/184 2026-02-13T02:19:24.658 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-osd-classic-2:20.3.0-5241.g8f8d3829.el9.x86 137/184 2026-02-13T02:19:24.720 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-osd-crimson-2:20.3.0-5241.g8f8d3829.el9.x86 138/184 2026-02-13T02:19:24.722 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-volume-2:20.3.0-5241.g8f8d3829.el9.noarch 139/184 2026-02-13T02:19:24.731 INFO:teuthology.orchestra.run.trial107.stdout: Installing : ceph-fuse-2:20.3.0-5241.g8f8d3829.el9.x86_64 166/184 2026-02-13T02:19:24.737 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-volume-2:20.3.0-5241.g8f8d3829.el9.noarch 139/184 2026-02-13T02:19:24.737 INFO:teuthology.orchestra.run.trial028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T02:19:24.737 INFO:teuthology.orchestra.run.trial028.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-13T02:19:24.737 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:19:24.748 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-osd-2:20.3.0-5241.g8f8d3829.el9.x86_64 140/184 2026-02-13T02:19:24.755 INFO:teuthology.orchestra.run.trial107.stdout: Installing : openssl-devel-1:3.5.5-1.el9.x86_64 167/184 2026-02-13T02:19:24.757 INFO:teuthology.orchestra.run.trial107.stdout: Installing : libtool-2.4.6-46.el9.x86_64 168/184 2026-02-13T02:19:24.760 INFO:teuthology.orchestra.run.trial107.stdout: Upgrading : NetworkManager-team-1:1.54.3-2.el9.x86_64 169/184 2026-02-13T02:19:24.762 INFO:teuthology.orchestra.run.trial107.stdout: Upgrading : NetworkManager-tui-1:1.54.3-2.el9.x86_64 170/184 2026-02-13T02:19:24.771 INFO:teuthology.orchestra.run.trial107.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 171/184 2026-02-13T02:19:24.773 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-osd-2:20.3.0-5241.g8f8d3829.el9.x86_64 140/184 2026-02-13T02:19:24.773 INFO:teuthology.orchestra.run.trial028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T02:19:24.773 INFO:teuthology.orchestra.run.trial028.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-13T02:19:24.773 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-13T02:19:24.773 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-13T02:19:24.773 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:19:24.798 INFO:teuthology.orchestra.run.trial107.stdout: Installing : flex-2.6.4-9.el9.x86_64 172/184 2026-02-13T02:19:24.800 INFO:teuthology.orchestra.run.trial107.stdout: Installing : bison-3.7.4-5.el9.x86_64 173/184 2026-02-13T02:19:24.802 INFO:teuthology.orchestra.run.trial107.stdout: Cleanup : NetworkManager-tui-1:1.54.1-1.el9.x86_64 174/184 2026-02-13T02:19:24.806 INFO:teuthology.orchestra.run.trial107.stdout: Cleanup : openssl-1:3.5.1-6.el9.x86_64 175/184 2026-02-13T02:19:24.806 INFO:teuthology.orchestra.run.trial107.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 176/184 2026-02-13T02:19:24.820 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 176/184 2026-02-13T02:19:24.820 INFO:teuthology.orchestra.run.trial107.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 177/184 2026-02-13T02:19:24.833 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 177/184 2026-02-13T02:19:24.834 INFO:teuthology.orchestra.run.trial107.stdout: Cleanup : openssl-fips-provider-1:3.5.1-6.el9.x86_64 178/184 2026-02-13T02:19:24.835 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: cephadm-2:20.3.0-5241.g8f8d3829.el9.noarch 141/184 2026-02-13T02:19:24.836 INFO:teuthology.orchestra.run.trial107.stdout: Cleanup : iproute-tc-6.17.0-1.el9.x86_64 179/184 2026-02-13T02:19:24.838 INFO:teuthology.orchestra.run.trial107.stdout: Cleanup : NetworkManager-team-1:1.54.1-1.el9.x86_64 180/184 2026-02-13T02:19:24.838 INFO:teuthology.orchestra.run.trial028.stdout: Installing : cephadm-2:20.3.0-5241.g8f8d3829.el9.noarch 141/184 2026-02-13T02:19:24.844 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: NetworkManager-1:1.54.1-1.el9.x86_64 181/184 2026-02-13T02:19:24.844 INFO:teuthology.orchestra.run.trial107.stdout: Cleanup : NetworkManager-1:1.54.1-1.el9.x86_64 181/184 2026-02-13T02:19:24.847 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5241.g8f8d3829.e 142/184 2026-02-13T02:19:24.870 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5241.g8f8d3829. 143/184 2026-02-13T02:19:24.872 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5241.g8f8d3829.el9.noa 144/184 2026-02-13T02:19:24.936 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: NetworkManager-1:1.54.1-1.el9.x86_64 181/184 2026-02-13T02:19:24.938 INFO:teuthology.orchestra.run.trial107.stdout: Cleanup : NetworkManager-libnm-1:1.54.1-1.el9.x86_64 182/184 2026-02-13T02:19:24.941 INFO:teuthology.orchestra.run.trial107.stdout: Cleanup : iproute-6.17.0-1.el9.x86_64 183/184 2026-02-13T02:19:24.942 INFO:teuthology.orchestra.run.trial107.stdout: Cleanup : openssl-libs-1:3.5.1-6.el9.x86_64 184/184 2026-02-13T02:19:24.948 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: iproute-6.17.0-2.el9.x86_64 184/184 2026-02-13T02:19:24.954 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5241.g8f8d3829.el9.x86 184/184 2026-02-13T02:19:24.979 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5241.g8f8d3829.el9.x86 184/184 2026-02-13T02:19:25.956 INFO:teuthology.orchestra.run.trial107.stdout: Running scriptlet: openssl-libs-1:3.5.1-6.el9.x86_64 184/184 2026-02-13T02:19:25.956 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : ceph-2:20.3.0-5241.g8f8d3829.el9.x86_64 1/184 2026-02-13T02:19:25.956 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : ceph-base-2:20.3.0-5241.g8f8d3829.el9.x86_64 2/184 2026-02-13T02:19:25.956 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : ceph-common-2:20.3.0-5241.g8f8d3829.el9.x86_64 3/184 2026-02-13T02:19:25.956 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : ceph-fuse-2:20.3.0-5241.g8f8d3829.el9.x86_64 4/184 2026-02-13T02:19:25.956 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5241.g8f8d3 5/184 2026-02-13T02:19:25.956 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : ceph-mds-2:20.3.0-5241.g8f8d3829.el9.x86_64 6/184 2026-02-13T02:19:25.957 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : ceph-mgr-2:20.3.0-5241.g8f8d3829.el9.x86_64 7/184 2026-02-13T02:19:25.957 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : ceph-mon-2:20.3.0-5241.g8f8d3829.el9.x86_64 8/184 2026-02-13T02:19:25.957 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : ceph-osd-2:20.3.0-5241.g8f8d3829.el9.x86_64 9/184 2026-02-13T02:19:25.957 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : ceph-osd-classic-2:20.3.0-5241.g8f8d3829.el9.x86 10/184 2026-02-13T02:19:25.957 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5241.g8f8d3829.el9.x86 11/184 2026-02-13T02:19:25.957 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : ceph-radosgw-2:20.3.0-5241.g8f8d3829.el9.x86_64 12/184 2026-02-13T02:19:25.957 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : ceph-selinux-2:20.3.0-5241.g8f8d3829.el9.x86_64 13/184 2026-02-13T02:19:25.957 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : ceph-test-2:20.3.0-5241.g8f8d3829.el9.x86_64 14/184 2026-02-13T02:19:25.957 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : cephfs-mirror-2:20.3.0-5241.g8f8d3829.el9.x86_64 15/184 2026-02-13T02:19:25.957 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : cephfs-shell-2:20.3.0-5241.g8f8d3829.el9.x86_64 16/184 2026-02-13T02:19:25.957 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : libcephfs-devel-2:20.3.0-5241.g8f8d3829.el9.x86_ 17/184 2026-02-13T02:19:25.957 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5241.g8f8d3829.el9.x86 18/184 2026-02-13T02:19:25.958 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : libcephfs2-2:20.3.0-5241.g8f8d3829.el9.x86_64 19/184 2026-02-13T02:19:25.958 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : libcephsqlite-2:20.3.0-5241.g8f8d3829.el9.x86_64 20/184 2026-02-13T02:19:25.958 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : librados-devel-2:20.3.0-5241.g8f8d3829.el9.x86_6 21/184 2026-02-13T02:19:25.958 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : libradosstriper1-2:20.3.0-5241.g8f8d3829.el9.x86 22/184 2026-02-13T02:19:25.958 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : librgw2-2:20.3.0-5241.g8f8d3829.el9.x86_64 23/184 2026-02-13T02:19:25.958 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5241.g8f8d3829.el 24/184 2026-02-13T02:19:25.958 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-ceph-common-2:20.3.0-5241.g8f8d3829.el9. 25/184 2026-02-13T02:19:25.958 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-cephfs-2:20.3.0-5241.g8f8d3829.el9.x86_6 26/184 2026-02-13T02:19:25.958 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-rados-2:20.3.0-5241.g8f8d3829.el9.x86_64 27/184 2026-02-13T02:19:25.958 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-rbd-2:20.3.0-5241.g8f8d3829.el9.x86_64 28/184 2026-02-13T02:19:25.958 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-rgw-2:20.3.0-5241.g8f8d3829.el9.x86_64 29/184 2026-02-13T02:19:25.958 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : rbd-fuse-2:20.3.0-5241.g8f8d3829.el9.x86_64 30/184 2026-02-13T02:19:25.959 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : rbd-mirror-2:20.3.0-5241.g8f8d3829.el9.x86_64 31/184 2026-02-13T02:19:25.959 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : rbd-nbd-2:20.3.0-5241.g8f8d3829.el9.x86_64 32/184 2026-02-13T02:19:25.959 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5241.g8f8d3829. 33/184 2026-02-13T02:19:25.959 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5241.g8f8d3829.el9.noa 34/184 2026-02-13T02:19:25.959 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5241.g8f8d3829.el9.n 35/184 2026-02-13T02:19:25.959 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5241.g8f8 36/184 2026-02-13T02:19:25.959 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5241.g8f8d3829.el 37/184 2026-02-13T02:19:25.959 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5241.g8f8d3829.el9.noarch 38/184 2026-02-13T02:19:25.959 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5241.g8f8d3829.e 39/184 2026-02-13T02:19:25.959 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : ceph-volume-2:20.3.0-5241.g8f8d3829.el9.noarch 40/184 2026-02-13T02:19:25.959 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : cephadm-2:20.3.0-5241.g8f8d3829.el9.noarch 41/184 2026-02-13T02:19:25.959 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : cephfs-top-2:20.3.0-5241.g8f8d3829.el9.noarch 42/184 2026-02-13T02:19:25.959 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : cryptsetup-2.8.1-2.el9.x86_64 43/184 2026-02-13T02:19:25.960 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 44/184 2026-02-13T02:19:25.960 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 45/184 2026-02-13T02:19:25.960 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 46/184 2026-02-13T02:19:25.960 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 47/184 2026-02-13T02:19:25.960 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 48/184 2026-02-13T02:19:25.960 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 49/184 2026-02-13T02:19:25.960 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 50/184 2026-02-13T02:19:25.960 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 51/184 2026-02-13T02:19:25.960 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 52/184 2026-02-13T02:19:25.960 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 53/184 2026-02-13T02:19:25.960 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 54/184 2026-02-13T02:19:25.960 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-ply-3.11-14.el9.noarch 55/184 2026-02-13T02:19:25.961 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 56/184 2026-02-13T02:19:25.961 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 57/184 2026-02-13T02:19:25.961 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 58/184 2026-02-13T02:19:25.961 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : unzip-6.0-59.el9.x86_64 59/184 2026-02-13T02:19:25.961 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : zip-3.0-35.el9.x86_64 60/184 2026-02-13T02:19:25.961 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : automake-1.16.2-8.el9.noarch 61/184 2026-02-13T02:19:25.961 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : bison-3.7.4-5.el9.x86_64 62/184 2026-02-13T02:19:25.961 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 63/184 2026-02-13T02:19:25.961 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : elfutils-libelf-devel-0.194-1.el9.x86_64 64/184 2026-02-13T02:19:25.961 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : flex-2.6.4-9.el9.x86_64 65/184 2026-02-13T02:19:25.961 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 66/184 2026-02-13T02:19:25.961 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 67/184 2026-02-13T02:19:25.962 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 68/184 2026-02-13T02:19:25.962 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 69/184 2026-02-13T02:19:25.962 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 70/184 2026-02-13T02:19:25.962 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 71/184 2026-02-13T02:19:25.962 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 72/184 2026-02-13T02:19:25.962 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 73/184 2026-02-13T02:19:25.962 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 74/184 2026-02-13T02:19:25.962 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 75/184 2026-02-13T02:19:25.962 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : libuuid-devel-2.37.4-21.el9.x86_64 76/184 2026-02-13T02:19:25.962 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 77/184 2026-02-13T02:19:25.962 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : libzstd-devel-1.5.5-1.el9.x86_64 78/184 2026-02-13T02:19:25.962 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 79/184 2026-02-13T02:19:25.963 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : lua-5.4.4-4.el9.x86_64 80/184 2026-02-13T02:19:25.963 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 81/184 2026-02-13T02:19:25.963 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 82/184 2026-02-13T02:19:25.963 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : openssl-devel-1:3.5.5-1.el9.x86_64 83/184 2026-02-13T02:19:25.963 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : perl-Thread-Queue-3.14-460.el9.noarch 84/184 2026-02-13T02:19:25.963 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : perl-threads-1:2.25-460.el9.x86_64 85/184 2026-02-13T02:19:25.963 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : perl-threads-shared-1.61-460.el9.x86_64 86/184 2026-02-13T02:19:25.963 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : postgresql-13.23-1.el9.x86_64 87/184 2026-02-13T02:19:25.963 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : postgresql-contrib-13.23-1.el9.x86_64 88/184 2026-02-13T02:19:25.963 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : postgresql-private-libs-13.23-1.el9.x86_64 89/184 2026-02-13T02:19:25.963 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : postgresql-server-13.23-1.el9.x86_64 90/184 2026-02-13T02:19:25.963 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 91/184 2026-02-13T02:19:25.963 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 92/184 2026-02-13T02:19:25.964 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 93/184 2026-02-13T02:19:25.964 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-devel-3.9.25-2.el9.x86_64 94/184 2026-02-13T02:19:25.964 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 95/184 2026-02-13T02:19:25.964 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 96/184 2026-02-13T02:19:25.964 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 97/184 2026-02-13T02:19:25.964 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 98/184 2026-02-13T02:19:25.964 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 99/184 2026-02-13T02:19:25.964 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 100/184 2026-02-13T02:19:25.964 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 101/184 2026-02-13T02:19:25.964 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 102/184 2026-02-13T02:19:25.964 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 103/184 2026-02-13T02:19:25.964 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 104/184 2026-02-13T02:19:25.965 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 105/184 2026-02-13T02:19:25.965 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 106/184 2026-02-13T02:19:25.965 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 107/184 2026-02-13T02:19:25.965 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 108/184 2026-02-13T02:19:25.965 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 109/184 2026-02-13T02:19:25.965 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 110/184 2026-02-13T02:19:25.965 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 111/184 2026-02-13T02:19:25.965 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 112/184 2026-02-13T02:19:25.965 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 113/184 2026-02-13T02:19:25.965 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : xfsprogs-devel-6.4.0-7.el9.x86_64 114/184 2026-02-13T02:19:25.965 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 115/184 2026-02-13T02:19:25.965 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 116/184 2026-02-13T02:19:25.966 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 117/184 2026-02-13T02:19:25.966 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 118/184 2026-02-13T02:19:25.966 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 119/184 2026-02-13T02:19:25.966 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 120/184 2026-02-13T02:19:25.966 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 121/184 2026-02-13T02:19:25.966 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 122/184 2026-02-13T02:19:25.966 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 123/184 2026-02-13T02:19:25.966 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 124/184 2026-02-13T02:19:25.966 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 125/184 2026-02-13T02:19:25.966 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 126/184 2026-02-13T02:19:25.966 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 127/184 2026-02-13T02:19:25.966 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 128/184 2026-02-13T02:19:25.966 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 129/184 2026-02-13T02:19:25.967 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 130/184 2026-02-13T02:19:25.967 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 131/184 2026-02-13T02:19:25.967 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 132/184 2026-02-13T02:19:25.967 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 133/184 2026-02-13T02:19:25.967 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 134/184 2026-02-13T02:19:25.967 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 135/184 2026-02-13T02:19:25.967 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 136/184 2026-02-13T02:19:25.967 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 137/184 2026-02-13T02:19:25.967 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 138/184 2026-02-13T02:19:25.967 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 139/184 2026-02-13T02:19:25.967 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 140/184 2026-02-13T02:19:25.967 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 141/184 2026-02-13T02:19:25.967 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 142/184 2026-02-13T02:19:25.968 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 143/184 2026-02-13T02:19:25.968 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 144/184 2026-02-13T02:19:25.968 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 145/184 2026-02-13T02:19:25.968 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 146/184 2026-02-13T02:19:25.968 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 147/184 2026-02-13T02:19:25.968 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 148/184 2026-02-13T02:19:25.968 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 149/184 2026-02-13T02:19:25.968 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 150/184 2026-02-13T02:19:25.968 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 151/184 2026-02-13T02:19:25.968 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 152/184 2026-02-13T02:19:25.968 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 153/184 2026-02-13T02:19:25.968 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 154/184 2026-02-13T02:19:25.968 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 155/184 2026-02-13T02:19:25.969 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 156/184 2026-02-13T02:19:25.969 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 157/184 2026-02-13T02:19:25.969 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 158/184 2026-02-13T02:19:25.969 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 159/184 2026-02-13T02:19:25.969 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : re2-1:20211101-20.el9.x86_64 160/184 2026-02-13T02:19:25.969 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 161/184 2026-02-13T02:19:25.969 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 162/184 2026-02-13T02:19:25.969 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : librados2-2:20.3.0-5241.g8f8d3829.el9.x86_64 163/184 2026-02-13T02:19:25.969 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 164/184 2026-02-13T02:19:25.969 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : librbd1-2:20.3.0-5241.g8f8d3829.el9.x86_64 165/184 2026-02-13T02:19:25.969 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 166/184 2026-02-13T02:19:25.969 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : NetworkManager-1:1.54.3-2.el9.x86_64 167/184 2026-02-13T02:19:25.969 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : NetworkManager-1:1.54.1-1.el9.x86_64 168/184 2026-02-13T02:19:25.970 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : NetworkManager-libnm-1:1.54.3-2.el9.x86_64 169/184 2026-02-13T02:19:25.970 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : NetworkManager-libnm-1:1.54.1-1.el9.x86_64 170/184 2026-02-13T02:19:25.970 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : NetworkManager-team-1:1.54.3-2.el9.x86_64 171/184 2026-02-13T02:19:25.970 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : NetworkManager-team-1:1.54.1-1.el9.x86_64 172/184 2026-02-13T02:19:25.970 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : NetworkManager-tui-1:1.54.3-2.el9.x86_64 173/184 2026-02-13T02:19:25.970 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : NetworkManager-tui-1:1.54.1-1.el9.x86_64 174/184 2026-02-13T02:19:25.970 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : iproute-6.17.0-2.el9.x86_64 175/184 2026-02-13T02:19:25.970 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : iproute-6.17.0-1.el9.x86_64 176/184 2026-02-13T02:19:25.970 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : iproute-tc-6.17.0-2.el9.x86_64 177/184 2026-02-13T02:19:25.970 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : iproute-tc-6.17.0-1.el9.x86_64 178/184 2026-02-13T02:19:25.970 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : openssl-1:3.5.5-1.el9.x86_64 179/184 2026-02-13T02:19:25.970 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : openssl-1:3.5.1-6.el9.x86_64 180/184 2026-02-13T02:19:25.971 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : openssl-fips-provider-1:3.5.5-1.el9.x86_64 181/184 2026-02-13T02:19:25.971 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : openssl-fips-provider-1:3.5.1-6.el9.x86_64 182/184 2026-02-13T02:19:25.971 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : openssl-libs-1:3.5.5-1.el9.x86_64 183/184 2026-02-13T02:19:26.024 INFO:teuthology.orchestra.run.trial107.stdout: Verifying : openssl-libs-1:3.5.1-6.el9.x86_64 184/184 2026-02-13T02:19:26.024 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:19:26.024 INFO:teuthology.orchestra.run.trial107.stdout:Upgraded: 2026-02-13T02:19:26.025 INFO:teuthology.orchestra.run.trial107.stdout: NetworkManager-1:1.54.3-2.el9.x86_64 2026-02-13T02:19:26.025 INFO:teuthology.orchestra.run.trial107.stdout: NetworkManager-libnm-1:1.54.3-2.el9.x86_64 2026-02-13T02:19:26.025 INFO:teuthology.orchestra.run.trial107.stdout: NetworkManager-team-1:1.54.3-2.el9.x86_64 2026-02-13T02:19:26.025 INFO:teuthology.orchestra.run.trial107.stdout: NetworkManager-tui-1:1.54.3-2.el9.x86_64 2026-02-13T02:19:26.025 INFO:teuthology.orchestra.run.trial107.stdout: iproute-6.17.0-2.el9.x86_64 2026-02-13T02:19:26.025 INFO:teuthology.orchestra.run.trial107.stdout: iproute-tc-6.17.0-2.el9.x86_64 2026-02-13T02:19:26.025 INFO:teuthology.orchestra.run.trial107.stdout: librados2-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.025 INFO:teuthology.orchestra.run.trial107.stdout: librbd1-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.025 INFO:teuthology.orchestra.run.trial107.stdout: openssl-1:3.5.5-1.el9.x86_64 2026-02-13T02:19:26.025 INFO:teuthology.orchestra.run.trial107.stdout: openssl-fips-provider-1:3.5.5-1.el9.x86_64 2026-02-13T02:19:26.025 INFO:teuthology.orchestra.run.trial107.stdout: openssl-libs-1:3.5.5-1.el9.x86_64 2026-02-13T02:19:26.025 INFO:teuthology.orchestra.run.trial107.stdout:Installed: 2026-02-13T02:19:26.026 INFO:teuthology.orchestra.run.trial107.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-13T02:19:26.026 INFO:teuthology.orchestra.run.trial107.stdout: automake-1.16.2-8.el9.noarch 2026-02-13T02:19:26.026 INFO:teuthology.orchestra.run.trial107.stdout: bison-3.7.4-5.el9.x86_64 2026-02-13T02:19:26.026 INFO:teuthology.orchestra.run.trial107.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-13T02:19:26.026 INFO:teuthology.orchestra.run.trial107.stdout: ceph-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.026 INFO:teuthology.orchestra.run.trial107.stdout: ceph-base-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.026 INFO:teuthology.orchestra.run.trial107.stdout: ceph-common-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.026 INFO:teuthology.orchestra.run.trial107.stdout: ceph-fuse-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.026 INFO:teuthology.orchestra.run.trial107.stdout: ceph-grafana-dashboards-2:20.3.0-5241.g8f8d3829.el9.noarch 2026-02-13T02:19:26.026 INFO:teuthology.orchestra.run.trial107.stdout: ceph-immutable-object-cache-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.026 INFO:teuthology.orchestra.run.trial107.stdout: ceph-mds-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.027 INFO:teuthology.orchestra.run.trial107.stdout: ceph-mgr-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.027 INFO:teuthology.orchestra.run.trial107.stdout: ceph-mgr-cephadm-2:20.3.0-5241.g8f8d3829.el9.noarch 2026-02-13T02:19:26.027 INFO:teuthology.orchestra.run.trial107.stdout: ceph-mgr-dashboard-2:20.3.0-5241.g8f8d3829.el9.noarch 2026-02-13T02:19:26.027 INFO:teuthology.orchestra.run.trial107.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5241.g8f8d3829.el9.noarch 2026-02-13T02:19:26.027 INFO:teuthology.orchestra.run.trial107.stdout: ceph-mgr-modules-core-2:20.3.0-5241.g8f8d3829.el9.noarch 2026-02-13T02:19:26.027 INFO:teuthology.orchestra.run.trial107.stdout: ceph-mgr-rook-2:20.3.0-5241.g8f8d3829.el9.noarch 2026-02-13T02:19:26.027 INFO:teuthology.orchestra.run.trial107.stdout: ceph-mon-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.027 INFO:teuthology.orchestra.run.trial107.stdout: ceph-osd-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.027 INFO:teuthology.orchestra.run.trial107.stdout: ceph-osd-classic-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.027 INFO:teuthology.orchestra.run.trial107.stdout: ceph-osd-crimson-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.027 INFO:teuthology.orchestra.run.trial107.stdout: ceph-prometheus-alerts-2:20.3.0-5241.g8f8d3829.el9.noarch 2026-02-13T02:19:26.027 INFO:teuthology.orchestra.run.trial107.stdout: ceph-radosgw-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.028 INFO:teuthology.orchestra.run.trial107.stdout: ceph-selinux-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.028 INFO:teuthology.orchestra.run.trial107.stdout: ceph-test-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.028 INFO:teuthology.orchestra.run.trial107.stdout: ceph-volume-2:20.3.0-5241.g8f8d3829.el9.noarch 2026-02-13T02:19:26.028 INFO:teuthology.orchestra.run.trial107.stdout: cephadm-2:20.3.0-5241.g8f8d3829.el9.noarch 2026-02-13T02:19:26.028 INFO:teuthology.orchestra.run.trial107.stdout: cephfs-mirror-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.028 INFO:teuthology.orchestra.run.trial107.stdout: cephfs-shell-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.028 INFO:teuthology.orchestra.run.trial107.stdout: cephfs-top-2:20.3.0-5241.g8f8d3829.el9.noarch 2026-02-13T02:19:26.028 INFO:teuthology.orchestra.run.trial107.stdout: cryptsetup-2.8.1-2.el9.x86_64 2026-02-13T02:19:26.028 INFO:teuthology.orchestra.run.trial107.stdout: elfutils-libelf-devel-0.194-1.el9.x86_64 2026-02-13T02:19:26.028 INFO:teuthology.orchestra.run.trial107.stdout: flex-2.6.4-9.el9.x86_64 2026-02-13T02:19:26.028 INFO:teuthology.orchestra.run.trial107.stdout: flexiblas-3.0.4-8.el9.x86_64 2026-02-13T02:19:26.028 INFO:teuthology.orchestra.run.trial107.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2026-02-13T02:19:26.028 INFO:teuthology.orchestra.run.trial107.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2026-02-13T02:19:26.029 INFO:teuthology.orchestra.run.trial107.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-13T02:19:26.029 INFO:teuthology.orchestra.run.trial107.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-13T02:19:26.029 INFO:teuthology.orchestra.run.trial107.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-13T02:19:26.029 INFO:teuthology.orchestra.run.trial107.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-13T02:19:26.029 INFO:teuthology.orchestra.run.trial107.stdout: libaio-devel-0.3.111-13.el9.x86_64 2026-02-13T02:19:26.029 INFO:teuthology.orchestra.run.trial107.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-13T02:19:26.029 INFO:teuthology.orchestra.run.trial107.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-13T02:19:26.029 INFO:teuthology.orchestra.run.trial107.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-13T02:19:26.029 INFO:teuthology.orchestra.run.trial107.stdout: libcephfs-devel-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.029 INFO:teuthology.orchestra.run.trial107.stdout: libcephfs-proxy2-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.029 INFO:teuthology.orchestra.run.trial107.stdout: libcephfs2-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.029 INFO:teuthology.orchestra.run.trial107.stdout: libcephsqlite-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.029 INFO:teuthology.orchestra.run.trial107.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-13T02:19:26.030 INFO:teuthology.orchestra.run.trial107.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-13T02:19:26.030 INFO:teuthology.orchestra.run.trial107.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-13T02:19:26.030 INFO:teuthology.orchestra.run.trial107.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-13T02:19:26.030 INFO:teuthology.orchestra.run.trial107.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-13T02:19:26.030 INFO:teuthology.orchestra.run.trial107.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-13T02:19:26.030 INFO:teuthology.orchestra.run.trial107.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-13T02:19:26.030 INFO:teuthology.orchestra.run.trial107.stdout: librados-devel-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.030 INFO:teuthology.orchestra.run.trial107.stdout: libradosstriper1-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.030 INFO:teuthology.orchestra.run.trial107.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-13T02:19:26.030 INFO:teuthology.orchestra.run.trial107.stdout: librgw2-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.030 INFO:teuthology.orchestra.run.trial107.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-13T02:19:26.030 INFO:teuthology.orchestra.run.trial107.stdout: libtool-2.4.6-46.el9.x86_64 2026-02-13T02:19:26.031 INFO:teuthology.orchestra.run.trial107.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-13T02:19:26.031 INFO:teuthology.orchestra.run.trial107.stdout: libuuid-devel-2.37.4-21.el9.x86_64 2026-02-13T02:19:26.031 INFO:teuthology.orchestra.run.trial107.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-13T02:19:26.031 INFO:teuthology.orchestra.run.trial107.stdout: libzstd-devel-1.5.5-1.el9.x86_64 2026-02-13T02:19:26.031 INFO:teuthology.orchestra.run.trial107.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-13T02:19:26.031 INFO:teuthology.orchestra.run.trial107.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-13T02:19:26.031 INFO:teuthology.orchestra.run.trial107.stdout: lua-5.4.4-4.el9.x86_64 2026-02-13T02:19:26.031 INFO:teuthology.orchestra.run.trial107.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-13T02:19:26.031 INFO:teuthology.orchestra.run.trial107.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-13T02:19:26.031 INFO:teuthology.orchestra.run.trial107.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-13T02:19:26.031 INFO:teuthology.orchestra.run.trial107.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-13T02:19:26.031 INFO:teuthology.orchestra.run.trial107.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-13T02:19:26.031 INFO:teuthology.orchestra.run.trial107.stdout: openssl-devel-1:3.5.5-1.el9.x86_64 2026-02-13T02:19:26.032 INFO:teuthology.orchestra.run.trial107.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-13T02:19:26.032 INFO:teuthology.orchestra.run.trial107.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-13T02:19:26.032 INFO:teuthology.orchestra.run.trial107.stdout: perl-Thread-Queue-3.14-460.el9.noarch 2026-02-13T02:19:26.032 INFO:teuthology.orchestra.run.trial107.stdout: perl-threads-1:2.25-460.el9.x86_64 2026-02-13T02:19:26.032 INFO:teuthology.orchestra.run.trial107.stdout: perl-threads-shared-1.61-460.el9.x86_64 2026-02-13T02:19:26.032 INFO:teuthology.orchestra.run.trial107.stdout: postgresql-13.23-1.el9.x86_64 2026-02-13T02:19:26.032 INFO:teuthology.orchestra.run.trial107.stdout: postgresql-contrib-13.23-1.el9.x86_64 2026-02-13T02:19:26.032 INFO:teuthology.orchestra.run.trial107.stdout: postgresql-private-libs-13.23-1.el9.x86_64 2026-02-13T02:19:26.032 INFO:teuthology.orchestra.run.trial107.stdout: postgresql-server-13.23-1.el9.x86_64 2026-02-13T02:19:26.032 INFO:teuthology.orchestra.run.trial107.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-13T02:19:26.032 INFO:teuthology.orchestra.run.trial107.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-13T02:19:26.032 INFO:teuthology.orchestra.run.trial107.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-13T02:19:26.032 INFO:teuthology.orchestra.run.trial107.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-13T02:19:26.033 INFO:teuthology.orchestra.run.trial107.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-13T02:19:26.033 INFO:teuthology.orchestra.run.trial107.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-13T02:19:26.033 INFO:teuthology.orchestra.run.trial107.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-13T02:19:26.033 INFO:teuthology.orchestra.run.trial107.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-13T02:19:26.033 INFO:teuthology.orchestra.run.trial107.stdout: python3-ceph-argparse-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.033 INFO:teuthology.orchestra.run.trial107.stdout: python3-ceph-common-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.033 INFO:teuthology.orchestra.run.trial107.stdout: python3-cephfs-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.033 INFO:teuthology.orchestra.run.trial107.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-13T02:19:26.033 INFO:teuthology.orchestra.run.trial107.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-13T02:19:26.033 INFO:teuthology.orchestra.run.trial107.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-13T02:19:26.033 INFO:teuthology.orchestra.run.trial107.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-13T02:19:26.034 INFO:teuthology.orchestra.run.trial107.stdout: python3-cmd2-2.4.2-2.el9.noarch 2026-02-13T02:19:26.034 INFO:teuthology.orchestra.run.trial107.stdout: python3-colorama-0.4.6-3.el9.noarch 2026-02-13T02:19:26.034 INFO:teuthology.orchestra.run.trial107.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-13T02:19:26.034 INFO:teuthology.orchestra.run.trial107.stdout: python3-devel-3.9.25-2.el9.x86_64 2026-02-13T02:19:26.034 INFO:teuthology.orchestra.run.trial107.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-13T02:19:26.034 INFO:teuthology.orchestra.run.trial107.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-13T02:19:26.034 INFO:teuthology.orchestra.run.trial107.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-13T02:19:26.034 INFO:teuthology.orchestra.run.trial107.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-13T02:19:26.034 INFO:teuthology.orchestra.run.trial107.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-13T02:19:26.034 INFO:teuthology.orchestra.run.trial107.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-13T02:19:26.034 INFO:teuthology.orchestra.run.trial107.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-13T02:19:26.034 INFO:teuthology.orchestra.run.trial107.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-13T02:19:26.034 INFO:teuthology.orchestra.run.trial107.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-13T02:19:26.034 INFO:teuthology.orchestra.run.trial107.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-13T02:19:26.035 INFO:teuthology.orchestra.run.trial107.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-13T02:19:26.035 INFO:teuthology.orchestra.run.trial107.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-13T02:19:26.035 INFO:teuthology.orchestra.run.trial107.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-13T02:19:26.035 INFO:teuthology.orchestra.run.trial107.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-13T02:19:26.035 INFO:teuthology.orchestra.run.trial107.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-13T02:19:26.035 INFO:teuthology.orchestra.run.trial107.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-13T02:19:26.035 INFO:teuthology.orchestra.run.trial107.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-13T02:19:26.035 INFO:teuthology.orchestra.run.trial107.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-13T02:19:26.035 INFO:teuthology.orchestra.run.trial107.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-13T02:19:26.035 INFO:teuthology.orchestra.run.trial107.stdout: python3-ply-3.11-14.el9.noarch 2026-02-13T02:19:26.035 INFO:teuthology.orchestra.run.trial107.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-13T02:19:26.035 INFO:teuthology.orchestra.run.trial107.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-13T02:19:26.035 INFO:teuthology.orchestra.run.trial107.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-13T02:19:26.036 INFO:teuthology.orchestra.run.trial107.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-13T02:19:26.036 INFO:teuthology.orchestra.run.trial107.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-13T02:19:26.036 INFO:teuthology.orchestra.run.trial107.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-13T02:19:26.036 INFO:teuthology.orchestra.run.trial107.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2026-02-13T02:19:26.036 INFO:teuthology.orchestra.run.trial107.stdout: python3-rados-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.036 INFO:teuthology.orchestra.run.trial107.stdout: python3-rbd-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.036 INFO:teuthology.orchestra.run.trial107.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-13T02:19:26.036 INFO:teuthology.orchestra.run.trial107.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-13T02:19:26.036 INFO:teuthology.orchestra.run.trial107.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-13T02:19:26.036 INFO:teuthology.orchestra.run.trial107.stdout: python3-rgw-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.036 INFO:teuthology.orchestra.run.trial107.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-13T02:19:26.036 INFO:teuthology.orchestra.run.trial107.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-13T02:19:26.036 INFO:teuthology.orchestra.run.trial107.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-13T02:19:26.037 INFO:teuthology.orchestra.run.trial107.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-13T02:19:26.037 INFO:teuthology.orchestra.run.trial107.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-13T02:19:26.037 INFO:teuthology.orchestra.run.trial107.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-13T02:19:26.037 INFO:teuthology.orchestra.run.trial107.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-13T02:19:26.037 INFO:teuthology.orchestra.run.trial107.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2026-02-13T02:19:26.037 INFO:teuthology.orchestra.run.trial107.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-13T02:19:26.037 INFO:teuthology.orchestra.run.trial107.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-13T02:19:26.037 INFO:teuthology.orchestra.run.trial107.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-13T02:19:26.037 INFO:teuthology.orchestra.run.trial107.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-13T02:19:26.037 INFO:teuthology.orchestra.run.trial107.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-13T02:19:26.037 INFO:teuthology.orchestra.run.trial107.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-13T02:19:26.037 INFO:teuthology.orchestra.run.trial107.stdout: rbd-fuse-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.037 INFO:teuthology.orchestra.run.trial107.stdout: rbd-mirror-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.038 INFO:teuthology.orchestra.run.trial107.stdout: rbd-nbd-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:26.038 INFO:teuthology.orchestra.run.trial107.stdout: re2-1:20211101-20.el9.x86_64 2026-02-13T02:19:26.038 INFO:teuthology.orchestra.run.trial107.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-13T02:19:26.038 INFO:teuthology.orchestra.run.trial107.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-13T02:19:26.038 INFO:teuthology.orchestra.run.trial107.stdout: unzip-6.0-59.el9.x86_64 2026-02-13T02:19:26.038 INFO:teuthology.orchestra.run.trial107.stdout: uuid-1.6.2-55.el9.x86_64 2026-02-13T02:19:26.038 INFO:teuthology.orchestra.run.trial107.stdout: xfsprogs-devel-6.4.0-7.el9.x86_64 2026-02-13T02:19:26.038 INFO:teuthology.orchestra.run.trial107.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-13T02:19:26.038 INFO:teuthology.orchestra.run.trial107.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-13T02:19:26.038 INFO:teuthology.orchestra.run.trial107.stdout: zip-3.0-35.el9.x86_64 2026-02-13T02:19:26.039 INFO:teuthology.orchestra.run.trial107.stdout: zlib-devel-1.2.11-41.el9.x86_64 2026-02-13T02:19:26.039 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:19:26.039 INFO:teuthology.orchestra.run.trial107.stdout:Complete! 2026-02-13T02:19:26.083 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5241.g8f8d3829.el9.noa 144/184 2026-02-13T02:19:26.090 INFO:teuthology.orchestra.run.trial027.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5241.g8f8d3829.el9.n 145/184 2026-02-13T02:19:26.123 DEBUG:teuthology.parallel:result is None 2026-02-13T02:19:26.448 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5241.g8f8d3829.el9.noa 144/184 2026-02-13T02:19:26.456 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5241.g8f8d3829.el9.n 145/184 2026-02-13T02:19:26.542 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5241.g8f8d3829.el9.n 145/184 2026-02-13T02:19:26.543 INFO:teuthology.orchestra.run.trial027.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5241.g8f8 146/184 2026-02-13T02:19:26.602 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5241.g8f8 146/184 2026-02-13T02:19:26.644 INFO:teuthology.orchestra.run.trial027.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5241.g8f8d3829.el 147/184 2026-02-13T02:19:26.646 INFO:teuthology.orchestra.run.trial027.stdout: Installing : ceph-mgr-2:20.3.0-5241.g8f8d3829.el9.x86_64 148/184 2026-02-13T02:19:26.666 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5241.g8f8d3829.el9.x86_64 148/184 2026-02-13T02:19:26.666 INFO:teuthology.orchestra.run.trial027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T02:19:26.666 INFO:teuthology.orchestra.run.trial027.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-13T02:19:26.666 INFO:teuthology.orchestra.run.trial027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-13T02:19:26.666 INFO:teuthology.orchestra.run.trial027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-13T02:19:26.666 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:19:26.677 INFO:teuthology.orchestra.run.trial027.stdout: Installing : ceph-mgr-rook-2:20.3.0-5241.g8f8d3829.el9.noarch 149/184 2026-02-13T02:19:26.686 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5241.g8f8d3829.el9.noarch 149/184 2026-02-13T02:19:26.915 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5241.g8f8d3829.el9.n 145/184 2026-02-13T02:19:26.916 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5241.g8f8 146/184 2026-02-13T02:19:26.978 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5241.g8f8 146/184 2026-02-13T02:19:27.020 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5241.g8f8d3829.el 147/184 2026-02-13T02:19:27.022 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-mgr-2:20.3.0-5241.g8f8d3829.el9.x86_64 148/184 2026-02-13T02:19:27.042 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5241.g8f8d3829.el9.x86_64 148/184 2026-02-13T02:19:27.042 INFO:teuthology.orchestra.run.trial028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T02:19:27.042 INFO:teuthology.orchestra.run.trial028.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-13T02:19:27.042 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-13T02:19:27.042 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-13T02:19:27.043 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:19:27.054 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-mgr-rook-2:20.3.0-5241.g8f8d3829.el9.noarch 149/184 2026-02-13T02:19:27.065 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5241.g8f8d3829.el9.noarch 149/184 2026-02-13T02:19:27.256 INFO:teuthology.orchestra.run.trial027.stdout: Installing : ceph-2:20.3.0-5241.g8f8d3829.el9.x86_64 150/184 2026-02-13T02:19:27.258 INFO:teuthology.orchestra.run.trial027.stdout: Installing : ceph-radosgw-2:20.3.0-5241.g8f8d3829.el9.x86_64 151/184 2026-02-13T02:19:27.279 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5241.g8f8d3829.el9.x86_64 151/184 2026-02-13T02:19:27.279 INFO:teuthology.orchestra.run.trial027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T02:19:27.279 INFO:teuthology.orchestra.run.trial027.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-13T02:19:27.279 INFO:teuthology.orchestra.run.trial027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-13T02:19:27.280 INFO:teuthology.orchestra.run.trial027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-13T02:19:27.280 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:19:27.288 INFO:teuthology.orchestra.run.trial027.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5241.g8f8d3 152/184 2026-02-13T02:19:27.308 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5241.g8f8d3 152/184 2026-02-13T02:19:27.308 INFO:teuthology.orchestra.run.trial027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T02:19:27.308 INFO:teuthology.orchestra.run.trial027.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-13T02:19:27.308 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:19:27.321 INFO:teuthology.orchestra.run.trial027.stdout: Installing : cephfs-mirror-2:20.3.0-5241.g8f8d3829.el9.x86_64 153/184 2026-02-13T02:19:27.342 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: cephfs-mirror-2:20.3.0-5241.g8f8d3829.el9.x86_64 153/184 2026-02-13T02:19:27.342 INFO:teuthology.orchestra.run.trial027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T02:19:27.342 INFO:teuthology.orchestra.run.trial027.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2026-02-13T02:19:27.342 INFO:teuthology.orchestra.run.trial027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2026-02-13T02:19:27.342 INFO:teuthology.orchestra.run.trial027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2026-02-13T02:19:27.342 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:19:27.466 INFO:teuthology.orchestra.run.trial027.stdout: Installing : rbd-mirror-2:20.3.0-5241.g8f8d3829.el9.x86_64 154/184 2026-02-13T02:19:27.487 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5241.g8f8d3829.el9.x86_64 154/184 2026-02-13T02:19:27.487 INFO:teuthology.orchestra.run.trial027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T02:19:27.487 INFO:teuthology.orchestra.run.trial027.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-13T02:19:27.487 INFO:teuthology.orchestra.run.trial027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-13T02:19:27.487 INFO:teuthology.orchestra.run.trial027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-13T02:19:27.487 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:19:27.657 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-2:20.3.0-5241.g8f8d3829.el9.x86_64 150/184 2026-02-13T02:19:27.658 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-radosgw-2:20.3.0-5241.g8f8d3829.el9.x86_64 151/184 2026-02-13T02:19:27.681 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5241.g8f8d3829.el9.x86_64 151/184 2026-02-13T02:19:27.681 INFO:teuthology.orchestra.run.trial028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T02:19:27.681 INFO:teuthology.orchestra.run.trial028.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-13T02:19:27.681 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-13T02:19:27.681 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-13T02:19:27.681 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:19:27.690 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5241.g8f8d3 152/184 2026-02-13T02:19:27.711 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5241.g8f8d3 152/184 2026-02-13T02:19:27.711 INFO:teuthology.orchestra.run.trial028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T02:19:27.711 INFO:teuthology.orchestra.run.trial028.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-13T02:19:27.711 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:19:27.725 INFO:teuthology.orchestra.run.trial028.stdout: Installing : cephfs-mirror-2:20.3.0-5241.g8f8d3829.el9.x86_64 153/184 2026-02-13T02:19:27.746 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: cephfs-mirror-2:20.3.0-5241.g8f8d3829.el9.x86_64 153/184 2026-02-13T02:19:27.746 INFO:teuthology.orchestra.run.trial028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T02:19:27.746 INFO:teuthology.orchestra.run.trial028.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2026-02-13T02:19:27.746 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2026-02-13T02:19:27.747 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2026-02-13T02:19:27.747 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:19:27.873 INFO:teuthology.orchestra.run.trial028.stdout: Installing : rbd-mirror-2:20.3.0-5241.g8f8d3829.el9.x86_64 154/184 2026-02-13T02:19:27.894 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5241.g8f8d3829.el9.x86_64 154/184 2026-02-13T02:19:27.894 INFO:teuthology.orchestra.run.trial028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-13T02:19:27.894 INFO:teuthology.orchestra.run.trial028.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-13T02:19:27.894 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-13T02:19:27.894 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-13T02:19:27.894 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:19:30.916 INFO:teuthology.orchestra.run.trial027.stdout: Installing : ceph-test-2:20.3.0-5241.g8f8d3829.el9.x86_64 155/184 2026-02-13T02:19:30.929 INFO:teuthology.orchestra.run.trial027.stdout: Installing : xfsprogs-devel-6.4.0-7.el9.x86_64 156/184 2026-02-13T02:19:30.932 INFO:teuthology.orchestra.run.trial027.stdout: Installing : elfutils-libelf-devel-0.194-1.el9.x86_64 157/184 2026-02-13T02:19:31.006 INFO:teuthology.orchestra.run.trial027.stdout: Installing : cephfs-shell-2:20.3.0-5241.g8f8d3829.el9.x86_64 158/184 2026-02-13T02:19:31.025 INFO:teuthology.orchestra.run.trial027.stdout: Installing : postgresql-contrib-13.23-1.el9.x86_64 159/184 2026-02-13T02:19:31.029 INFO:teuthology.orchestra.run.trial027.stdout: Upgrading : iproute-tc-6.17.0-2.el9.x86_64 160/184 2026-02-13T02:19:31.030 INFO:teuthology.orchestra.run.trial027.stdout: Installing : libcephfs-devel-2:20.3.0-5241.g8f8d3829.el9.x86_ 161/184 2026-02-13T02:19:31.152 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: postgresql-server-13.23-1.el9.x86_64 162/184 2026-02-13T02:19:31.156 INFO:teuthology.orchestra.run.trial027.stdout: Installing : postgresql-server-13.23-1.el9.x86_64 162/184 2026-02-13T02:19:31.174 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: postgresql-server-13.23-1.el9.x86_64 162/184 2026-02-13T02:19:31.179 INFO:teuthology.orchestra.run.trial027.stdout: Installing : cephfs-top-2:20.3.0-5241.g8f8d3829.el9.noarch 163/184 2026-02-13T02:19:31.188 INFO:teuthology.orchestra.run.trial027.stdout: Installing : rbd-fuse-2:20.3.0-5241.g8f8d3829.el9.x86_64 164/184 2026-02-13T02:19:31.231 INFO:teuthology.orchestra.run.trial027.stdout: Installing : rbd-nbd-2:20.3.0-5241.g8f8d3829.el9.x86_64 165/184 2026-02-13T02:19:31.421 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-test-2:20.3.0-5241.g8f8d3829.el9.x86_64 155/184 2026-02-13T02:19:31.435 INFO:teuthology.orchestra.run.trial028.stdout: Installing : xfsprogs-devel-6.4.0-7.el9.x86_64 156/184 2026-02-13T02:19:31.439 INFO:teuthology.orchestra.run.trial028.stdout: Installing : elfutils-libelf-devel-0.194-1.el9.x86_64 157/184 2026-02-13T02:19:31.514 INFO:teuthology.orchestra.run.trial028.stdout: Installing : cephfs-shell-2:20.3.0-5241.g8f8d3829.el9.x86_64 158/184 2026-02-13T02:19:31.533 INFO:teuthology.orchestra.run.trial028.stdout: Installing : postgresql-contrib-13.23-1.el9.x86_64 159/184 2026-02-13T02:19:31.537 INFO:teuthology.orchestra.run.trial028.stdout: Upgrading : iproute-tc-6.17.0-2.el9.x86_64 160/184 2026-02-13T02:19:31.539 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libcephfs-devel-2:20.3.0-5241.g8f8d3829.el9.x86_ 161/184 2026-02-13T02:19:31.664 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: postgresql-server-13.23-1.el9.x86_64 162/184 2026-02-13T02:19:31.668 INFO:teuthology.orchestra.run.trial028.stdout: Installing : postgresql-server-13.23-1.el9.x86_64 162/184 2026-02-13T02:19:31.682 INFO:teuthology.orchestra.run.trial027.stdout: Installing : ceph-fuse-2:20.3.0-5241.g8f8d3829.el9.x86_64 166/184 2026-02-13T02:19:31.685 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: postgresql-server-13.23-1.el9.x86_64 162/184 2026-02-13T02:19:31.690 INFO:teuthology.orchestra.run.trial028.stdout: Installing : cephfs-top-2:20.3.0-5241.g8f8d3829.el9.noarch 163/184 2026-02-13T02:19:31.701 INFO:teuthology.orchestra.run.trial028.stdout: Installing : rbd-fuse-2:20.3.0-5241.g8f8d3829.el9.x86_64 164/184 2026-02-13T02:19:31.707 INFO:teuthology.orchestra.run.trial027.stdout: Installing : openssl-devel-1:3.5.5-1.el9.x86_64 167/184 2026-02-13T02:19:31.709 INFO:teuthology.orchestra.run.trial027.stdout: Installing : libtool-2.4.6-46.el9.x86_64 168/184 2026-02-13T02:19:31.712 INFO:teuthology.orchestra.run.trial027.stdout: Upgrading : NetworkManager-team-1:1.54.3-2.el9.x86_64 169/184 2026-02-13T02:19:31.714 INFO:teuthology.orchestra.run.trial027.stdout: Upgrading : NetworkManager-tui-1:1.54.3-2.el9.x86_64 170/184 2026-02-13T02:19:31.723 INFO:teuthology.orchestra.run.trial027.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 171/184 2026-02-13T02:19:31.744 INFO:teuthology.orchestra.run.trial028.stdout: Installing : rbd-nbd-2:20.3.0-5241.g8f8d3829.el9.x86_64 165/184 2026-02-13T02:19:31.749 INFO:teuthology.orchestra.run.trial027.stdout: Installing : flex-2.6.4-9.el9.x86_64 172/184 2026-02-13T02:19:31.752 INFO:teuthology.orchestra.run.trial027.stdout: Installing : bison-3.7.4-5.el9.x86_64 173/184 2026-02-13T02:19:31.754 INFO:teuthology.orchestra.run.trial027.stdout: Cleanup : NetworkManager-tui-1:1.54.1-1.el9.x86_64 174/184 2026-02-13T02:19:31.757 INFO:teuthology.orchestra.run.trial027.stdout: Cleanup : openssl-1:3.5.1-6.el9.x86_64 175/184 2026-02-13T02:19:31.758 INFO:teuthology.orchestra.run.trial027.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 176/184 2026-02-13T02:19:31.770 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 176/184 2026-02-13T02:19:31.770 INFO:teuthology.orchestra.run.trial027.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 177/184 2026-02-13T02:19:31.783 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 177/184 2026-02-13T02:19:31.785 INFO:teuthology.orchestra.run.trial027.stdout: Cleanup : openssl-fips-provider-1:3.5.1-6.el9.x86_64 178/184 2026-02-13T02:19:31.787 INFO:teuthology.orchestra.run.trial027.stdout: Cleanup : iproute-tc-6.17.0-1.el9.x86_64 179/184 2026-02-13T02:19:31.788 INFO:teuthology.orchestra.run.trial027.stdout: Cleanup : NetworkManager-team-1:1.54.1-1.el9.x86_64 180/184 2026-02-13T02:19:31.795 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: NetworkManager-1:1.54.1-1.el9.x86_64 181/184 2026-02-13T02:19:31.795 INFO:teuthology.orchestra.run.trial027.stdout: Cleanup : NetworkManager-1:1.54.1-1.el9.x86_64 181/184 2026-02-13T02:19:31.886 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: NetworkManager-1:1.54.1-1.el9.x86_64 181/184 2026-02-13T02:19:31.888 INFO:teuthology.orchestra.run.trial027.stdout: Cleanup : NetworkManager-libnm-1:1.54.1-1.el9.x86_64 182/184 2026-02-13T02:19:31.891 INFO:teuthology.orchestra.run.trial027.stdout: Cleanup : iproute-6.17.0-1.el9.x86_64 183/184 2026-02-13T02:19:31.892 INFO:teuthology.orchestra.run.trial027.stdout: Cleanup : openssl-libs-1:3.5.1-6.el9.x86_64 184/184 2026-02-13T02:19:31.898 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: iproute-6.17.0-2.el9.x86_64 184/184 2026-02-13T02:19:31.904 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5241.g8f8d3829.el9.x86 184/184 2026-02-13T02:19:31.933 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5241.g8f8d3829.el9.x86 184/184 2026-02-13T02:19:32.200 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-fuse-2:20.3.0-5241.g8f8d3829.el9.x86_64 166/184 2026-02-13T02:19:32.225 INFO:teuthology.orchestra.run.trial028.stdout: Installing : openssl-devel-1:3.5.5-1.el9.x86_64 167/184 2026-02-13T02:19:32.227 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libtool-2.4.6-46.el9.x86_64 168/184 2026-02-13T02:19:32.230 INFO:teuthology.orchestra.run.trial028.stdout: Upgrading : NetworkManager-team-1:1.54.3-2.el9.x86_64 169/184 2026-02-13T02:19:32.232 INFO:teuthology.orchestra.run.trial028.stdout: Upgrading : NetworkManager-tui-1:1.54.3-2.el9.x86_64 170/184 2026-02-13T02:19:32.241 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 171/184 2026-02-13T02:19:32.268 INFO:teuthology.orchestra.run.trial028.stdout: Installing : flex-2.6.4-9.el9.x86_64 172/184 2026-02-13T02:19:32.271 INFO:teuthology.orchestra.run.trial028.stdout: Installing : bison-3.7.4-5.el9.x86_64 173/184 2026-02-13T02:19:32.273 INFO:teuthology.orchestra.run.trial028.stdout: Cleanup : NetworkManager-tui-1:1.54.1-1.el9.x86_64 174/184 2026-02-13T02:19:32.277 INFO:teuthology.orchestra.run.trial028.stdout: Cleanup : openssl-1:3.5.1-6.el9.x86_64 175/184 2026-02-13T02:19:32.277 INFO:teuthology.orchestra.run.trial028.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 176/184 2026-02-13T02:19:32.291 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 176/184 2026-02-13T02:19:32.291 INFO:teuthology.orchestra.run.trial028.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 177/184 2026-02-13T02:19:32.304 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 177/184 2026-02-13T02:19:32.306 INFO:teuthology.orchestra.run.trial028.stdout: Cleanup : openssl-fips-provider-1:3.5.1-6.el9.x86_64 178/184 2026-02-13T02:19:32.308 INFO:teuthology.orchestra.run.trial028.stdout: Cleanup : iproute-tc-6.17.0-1.el9.x86_64 179/184 2026-02-13T02:19:32.309 INFO:teuthology.orchestra.run.trial028.stdout: Cleanup : NetworkManager-team-1:1.54.1-1.el9.x86_64 180/184 2026-02-13T02:19:32.316 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: NetworkManager-1:1.54.1-1.el9.x86_64 181/184 2026-02-13T02:19:32.316 INFO:teuthology.orchestra.run.trial028.stdout: Cleanup : NetworkManager-1:1.54.1-1.el9.x86_64 181/184 2026-02-13T02:19:32.408 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: NetworkManager-1:1.54.1-1.el9.x86_64 181/184 2026-02-13T02:19:32.410 INFO:teuthology.orchestra.run.trial028.stdout: Cleanup : NetworkManager-libnm-1:1.54.1-1.el9.x86_64 182/184 2026-02-13T02:19:32.413 INFO:teuthology.orchestra.run.trial028.stdout: Cleanup : iproute-6.17.0-1.el9.x86_64 183/184 2026-02-13T02:19:32.415 INFO:teuthology.orchestra.run.trial028.stdout: Cleanup : openssl-libs-1:3.5.1-6.el9.x86_64 184/184 2026-02-13T02:19:32.421 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: iproute-6.17.0-2.el9.x86_64 184/184 2026-02-13T02:19:32.429 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5241.g8f8d3829.el9.x86 184/184 2026-02-13T02:19:32.454 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5241.g8f8d3829.el9.x86 184/184 2026-02-13T02:19:32.904 INFO:teuthology.orchestra.run.trial027.stdout: Running scriptlet: openssl-libs-1:3.5.1-6.el9.x86_64 184/184 2026-02-13T02:19:32.904 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : ceph-2:20.3.0-5241.g8f8d3829.el9.x86_64 1/184 2026-02-13T02:19:32.904 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : ceph-base-2:20.3.0-5241.g8f8d3829.el9.x86_64 2/184 2026-02-13T02:19:32.904 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : ceph-common-2:20.3.0-5241.g8f8d3829.el9.x86_64 3/184 2026-02-13T02:19:32.904 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : ceph-fuse-2:20.3.0-5241.g8f8d3829.el9.x86_64 4/184 2026-02-13T02:19:32.904 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5241.g8f8d3 5/184 2026-02-13T02:19:32.904 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : ceph-mds-2:20.3.0-5241.g8f8d3829.el9.x86_64 6/184 2026-02-13T02:19:32.904 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : ceph-mgr-2:20.3.0-5241.g8f8d3829.el9.x86_64 7/184 2026-02-13T02:19:32.904 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : ceph-mon-2:20.3.0-5241.g8f8d3829.el9.x86_64 8/184 2026-02-13T02:19:32.904 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : ceph-osd-2:20.3.0-5241.g8f8d3829.el9.x86_64 9/184 2026-02-13T02:19:32.904 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : ceph-osd-classic-2:20.3.0-5241.g8f8d3829.el9.x86 10/184 2026-02-13T02:19:32.904 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5241.g8f8d3829.el9.x86 11/184 2026-02-13T02:19:32.904 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : ceph-radosgw-2:20.3.0-5241.g8f8d3829.el9.x86_64 12/184 2026-02-13T02:19:32.904 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : ceph-selinux-2:20.3.0-5241.g8f8d3829.el9.x86_64 13/184 2026-02-13T02:19:32.904 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : ceph-test-2:20.3.0-5241.g8f8d3829.el9.x86_64 14/184 2026-02-13T02:19:32.904 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : cephfs-mirror-2:20.3.0-5241.g8f8d3829.el9.x86_64 15/184 2026-02-13T02:19:32.904 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : cephfs-shell-2:20.3.0-5241.g8f8d3829.el9.x86_64 16/184 2026-02-13T02:19:32.904 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : libcephfs-devel-2:20.3.0-5241.g8f8d3829.el9.x86_ 17/184 2026-02-13T02:19:32.904 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5241.g8f8d3829.el9.x86 18/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : libcephfs2-2:20.3.0-5241.g8f8d3829.el9.x86_64 19/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : libcephsqlite-2:20.3.0-5241.g8f8d3829.el9.x86_64 20/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : librados-devel-2:20.3.0-5241.g8f8d3829.el9.x86_6 21/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : libradosstriper1-2:20.3.0-5241.g8f8d3829.el9.x86 22/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : librgw2-2:20.3.0-5241.g8f8d3829.el9.x86_64 23/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5241.g8f8d3829.el 24/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-ceph-common-2:20.3.0-5241.g8f8d3829.el9. 25/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-cephfs-2:20.3.0-5241.g8f8d3829.el9.x86_6 26/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-rados-2:20.3.0-5241.g8f8d3829.el9.x86_64 27/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-rbd-2:20.3.0-5241.g8f8d3829.el9.x86_64 28/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-rgw-2:20.3.0-5241.g8f8d3829.el9.x86_64 29/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : rbd-fuse-2:20.3.0-5241.g8f8d3829.el9.x86_64 30/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : rbd-mirror-2:20.3.0-5241.g8f8d3829.el9.x86_64 31/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : rbd-nbd-2:20.3.0-5241.g8f8d3829.el9.x86_64 32/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5241.g8f8d3829. 33/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5241.g8f8d3829.el9.noa 34/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5241.g8f8d3829.el9.n 35/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5241.g8f8 36/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5241.g8f8d3829.el 37/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5241.g8f8d3829.el9.noarch 38/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5241.g8f8d3829.e 39/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : ceph-volume-2:20.3.0-5241.g8f8d3829.el9.noarch 40/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : cephadm-2:20.3.0-5241.g8f8d3829.el9.noarch 41/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : cephfs-top-2:20.3.0-5241.g8f8d3829.el9.noarch 42/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : cryptsetup-2.8.1-2.el9.x86_64 43/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 44/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 45/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 46/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 47/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 48/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 49/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 50/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 51/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 52/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 53/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 54/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-ply-3.11-14.el9.noarch 55/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 56/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 57/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 58/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : unzip-6.0-59.el9.x86_64 59/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : zip-3.0-35.el9.x86_64 60/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : automake-1.16.2-8.el9.noarch 61/184 2026-02-13T02:19:32.905 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : bison-3.7.4-5.el9.x86_64 62/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 63/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : elfutils-libelf-devel-0.194-1.el9.x86_64 64/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : flex-2.6.4-9.el9.x86_64 65/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 66/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 67/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 68/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 69/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 70/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 71/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 72/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 73/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 74/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 75/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : libuuid-devel-2.37.4-21.el9.x86_64 76/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 77/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : libzstd-devel-1.5.5-1.el9.x86_64 78/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 79/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : lua-5.4.4-4.el9.x86_64 80/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 81/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 82/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : openssl-devel-1:3.5.5-1.el9.x86_64 83/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : perl-Thread-Queue-3.14-460.el9.noarch 84/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : perl-threads-1:2.25-460.el9.x86_64 85/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : perl-threads-shared-1.61-460.el9.x86_64 86/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : postgresql-13.23-1.el9.x86_64 87/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : postgresql-contrib-13.23-1.el9.x86_64 88/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : postgresql-private-libs-13.23-1.el9.x86_64 89/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : postgresql-server-13.23-1.el9.x86_64 90/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 91/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 92/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 93/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-devel-3.9.25-2.el9.x86_64 94/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 95/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 96/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 97/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 98/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 99/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 100/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 101/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 102/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 103/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 104/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 105/184 2026-02-13T02:19:32.906 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 106/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 107/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 108/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 109/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 110/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 111/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 112/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 113/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : xfsprogs-devel-6.4.0-7.el9.x86_64 114/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 115/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 116/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 117/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 118/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 119/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 120/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 121/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 122/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 123/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 124/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 125/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 126/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 127/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 128/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 129/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 130/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 131/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 132/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 133/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 134/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 135/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 136/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 137/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 138/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 139/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 140/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 141/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 142/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 143/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 144/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 145/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 146/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 147/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 148/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 149/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 150/184 2026-02-13T02:19:32.907 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 151/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 152/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 153/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 154/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 155/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 156/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 157/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 158/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 159/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : re2-1:20211101-20.el9.x86_64 160/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 161/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 162/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : librados2-2:20.3.0-5241.g8f8d3829.el9.x86_64 163/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 164/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : librbd1-2:20.3.0-5241.g8f8d3829.el9.x86_64 165/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 166/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : NetworkManager-1:1.54.3-2.el9.x86_64 167/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : NetworkManager-1:1.54.1-1.el9.x86_64 168/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : NetworkManager-libnm-1:1.54.3-2.el9.x86_64 169/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : NetworkManager-libnm-1:1.54.1-1.el9.x86_64 170/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : NetworkManager-team-1:1.54.3-2.el9.x86_64 171/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : NetworkManager-team-1:1.54.1-1.el9.x86_64 172/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : NetworkManager-tui-1:1.54.3-2.el9.x86_64 173/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : NetworkManager-tui-1:1.54.1-1.el9.x86_64 174/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : iproute-6.17.0-2.el9.x86_64 175/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : iproute-6.17.0-1.el9.x86_64 176/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : iproute-tc-6.17.0-2.el9.x86_64 177/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : iproute-tc-6.17.0-1.el9.x86_64 178/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : openssl-1:3.5.5-1.el9.x86_64 179/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : openssl-1:3.5.1-6.el9.x86_64 180/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : openssl-fips-provider-1:3.5.5-1.el9.x86_64 181/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : openssl-fips-provider-1:3.5.1-6.el9.x86_64 182/184 2026-02-13T02:19:32.908 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : openssl-libs-1:3.5.5-1.el9.x86_64 183/184 2026-02-13T02:19:32.985 INFO:teuthology.orchestra.run.trial027.stdout: Verifying : openssl-libs-1:3.5.1-6.el9.x86_64 184/184 2026-02-13T02:19:32.985 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:19:32.985 INFO:teuthology.orchestra.run.trial027.stdout:Upgraded: 2026-02-13T02:19:32.985 INFO:teuthology.orchestra.run.trial027.stdout: NetworkManager-1:1.54.3-2.el9.x86_64 2026-02-13T02:19:32.986 INFO:teuthology.orchestra.run.trial027.stdout: NetworkManager-libnm-1:1.54.3-2.el9.x86_64 2026-02-13T02:19:32.986 INFO:teuthology.orchestra.run.trial027.stdout: NetworkManager-team-1:1.54.3-2.el9.x86_64 2026-02-13T02:19:32.986 INFO:teuthology.orchestra.run.trial027.stdout: NetworkManager-tui-1:1.54.3-2.el9.x86_64 2026-02-13T02:19:32.986 INFO:teuthology.orchestra.run.trial027.stdout: iproute-6.17.0-2.el9.x86_64 2026-02-13T02:19:32.986 INFO:teuthology.orchestra.run.trial027.stdout: iproute-tc-6.17.0-2.el9.x86_64 2026-02-13T02:19:32.986 INFO:teuthology.orchestra.run.trial027.stdout: librados2-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.986 INFO:teuthology.orchestra.run.trial027.stdout: librbd1-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.986 INFO:teuthology.orchestra.run.trial027.stdout: openssl-1:3.5.5-1.el9.x86_64 2026-02-13T02:19:32.986 INFO:teuthology.orchestra.run.trial027.stdout: openssl-fips-provider-1:3.5.5-1.el9.x86_64 2026-02-13T02:19:32.986 INFO:teuthology.orchestra.run.trial027.stdout: openssl-libs-1:3.5.5-1.el9.x86_64 2026-02-13T02:19:32.987 INFO:teuthology.orchestra.run.trial027.stdout:Installed: 2026-02-13T02:19:32.987 INFO:teuthology.orchestra.run.trial027.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-13T02:19:32.987 INFO:teuthology.orchestra.run.trial027.stdout: automake-1.16.2-8.el9.noarch 2026-02-13T02:19:32.987 INFO:teuthology.orchestra.run.trial027.stdout: bison-3.7.4-5.el9.x86_64 2026-02-13T02:19:32.987 INFO:teuthology.orchestra.run.trial027.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-13T02:19:32.987 INFO:teuthology.orchestra.run.trial027.stdout: ceph-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.987 INFO:teuthology.orchestra.run.trial027.stdout: ceph-base-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.987 INFO:teuthology.orchestra.run.trial027.stdout: ceph-common-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.987 INFO:teuthology.orchestra.run.trial027.stdout: ceph-fuse-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.987 INFO:teuthology.orchestra.run.trial027.stdout: ceph-grafana-dashboards-2:20.3.0-5241.g8f8d3829.el9.noarch 2026-02-13T02:19:32.987 INFO:teuthology.orchestra.run.trial027.stdout: ceph-immutable-object-cache-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.988 INFO:teuthology.orchestra.run.trial027.stdout: ceph-mds-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.988 INFO:teuthology.orchestra.run.trial027.stdout: ceph-mgr-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.988 INFO:teuthology.orchestra.run.trial027.stdout: ceph-mgr-cephadm-2:20.3.0-5241.g8f8d3829.el9.noarch 2026-02-13T02:19:32.988 INFO:teuthology.orchestra.run.trial027.stdout: ceph-mgr-dashboard-2:20.3.0-5241.g8f8d3829.el9.noarch 2026-02-13T02:19:32.988 INFO:teuthology.orchestra.run.trial027.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5241.g8f8d3829.el9.noarch 2026-02-13T02:19:32.988 INFO:teuthology.orchestra.run.trial027.stdout: ceph-mgr-modules-core-2:20.3.0-5241.g8f8d3829.el9.noarch 2026-02-13T02:19:32.988 INFO:teuthology.orchestra.run.trial027.stdout: ceph-mgr-rook-2:20.3.0-5241.g8f8d3829.el9.noarch 2026-02-13T02:19:32.988 INFO:teuthology.orchestra.run.trial027.stdout: ceph-mon-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.988 INFO:teuthology.orchestra.run.trial027.stdout: ceph-osd-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.988 INFO:teuthology.orchestra.run.trial027.stdout: ceph-osd-classic-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.988 INFO:teuthology.orchestra.run.trial027.stdout: ceph-osd-crimson-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.988 INFO:teuthology.orchestra.run.trial027.stdout: ceph-prometheus-alerts-2:20.3.0-5241.g8f8d3829.el9.noarch 2026-02-13T02:19:32.989 INFO:teuthology.orchestra.run.trial027.stdout: ceph-radosgw-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.989 INFO:teuthology.orchestra.run.trial027.stdout: ceph-selinux-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.989 INFO:teuthology.orchestra.run.trial027.stdout: ceph-test-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.989 INFO:teuthology.orchestra.run.trial027.stdout: ceph-volume-2:20.3.0-5241.g8f8d3829.el9.noarch 2026-02-13T02:19:32.989 INFO:teuthology.orchestra.run.trial027.stdout: cephadm-2:20.3.0-5241.g8f8d3829.el9.noarch 2026-02-13T02:19:32.989 INFO:teuthology.orchestra.run.trial027.stdout: cephfs-mirror-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.989 INFO:teuthology.orchestra.run.trial027.stdout: cephfs-shell-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.989 INFO:teuthology.orchestra.run.trial027.stdout: cephfs-top-2:20.3.0-5241.g8f8d3829.el9.noarch 2026-02-13T02:19:32.989 INFO:teuthology.orchestra.run.trial027.stdout: cryptsetup-2.8.1-2.el9.x86_64 2026-02-13T02:19:32.989 INFO:teuthology.orchestra.run.trial027.stdout: elfutils-libelf-devel-0.194-1.el9.x86_64 2026-02-13T02:19:32.989 INFO:teuthology.orchestra.run.trial027.stdout: flex-2.6.4-9.el9.x86_64 2026-02-13T02:19:32.989 INFO:teuthology.orchestra.run.trial027.stdout: flexiblas-3.0.4-8.el9.x86_64 2026-02-13T02:19:32.989 INFO:teuthology.orchestra.run.trial027.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2026-02-13T02:19:32.990 INFO:teuthology.orchestra.run.trial027.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2026-02-13T02:19:32.990 INFO:teuthology.orchestra.run.trial027.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-13T02:19:32.990 INFO:teuthology.orchestra.run.trial027.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-13T02:19:32.990 INFO:teuthology.orchestra.run.trial027.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-13T02:19:32.990 INFO:teuthology.orchestra.run.trial027.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-13T02:19:32.990 INFO:teuthology.orchestra.run.trial027.stdout: libaio-devel-0.3.111-13.el9.x86_64 2026-02-13T02:19:32.990 INFO:teuthology.orchestra.run.trial027.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-13T02:19:32.990 INFO:teuthology.orchestra.run.trial027.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-13T02:19:32.990 INFO:teuthology.orchestra.run.trial027.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-13T02:19:32.990 INFO:teuthology.orchestra.run.trial027.stdout: libcephfs-devel-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.990 INFO:teuthology.orchestra.run.trial027.stdout: libcephfs-proxy2-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.990 INFO:teuthology.orchestra.run.trial027.stdout: libcephfs2-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.991 INFO:teuthology.orchestra.run.trial027.stdout: libcephsqlite-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.991 INFO:teuthology.orchestra.run.trial027.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-13T02:19:32.991 INFO:teuthology.orchestra.run.trial027.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-13T02:19:32.991 INFO:teuthology.orchestra.run.trial027.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-13T02:19:32.991 INFO:teuthology.orchestra.run.trial027.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-13T02:19:32.991 INFO:teuthology.orchestra.run.trial027.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-13T02:19:32.991 INFO:teuthology.orchestra.run.trial027.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-13T02:19:32.991 INFO:teuthology.orchestra.run.trial027.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-13T02:19:32.991 INFO:teuthology.orchestra.run.trial027.stdout: librados-devel-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.991 INFO:teuthology.orchestra.run.trial027.stdout: libradosstriper1-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.991 INFO:teuthology.orchestra.run.trial027.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-13T02:19:32.991 INFO:teuthology.orchestra.run.trial027.stdout: librgw2-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.992 INFO:teuthology.orchestra.run.trial027.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-13T02:19:32.992 INFO:teuthology.orchestra.run.trial027.stdout: libtool-2.4.6-46.el9.x86_64 2026-02-13T02:19:32.992 INFO:teuthology.orchestra.run.trial027.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-13T02:19:32.992 INFO:teuthology.orchestra.run.trial027.stdout: libuuid-devel-2.37.4-21.el9.x86_64 2026-02-13T02:19:32.992 INFO:teuthology.orchestra.run.trial027.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-13T02:19:32.992 INFO:teuthology.orchestra.run.trial027.stdout: libzstd-devel-1.5.5-1.el9.x86_64 2026-02-13T02:19:32.992 INFO:teuthology.orchestra.run.trial027.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-13T02:19:32.992 INFO:teuthology.orchestra.run.trial027.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-13T02:19:32.992 INFO:teuthology.orchestra.run.trial027.stdout: lua-5.4.4-4.el9.x86_64 2026-02-13T02:19:32.992 INFO:teuthology.orchestra.run.trial027.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-13T02:19:32.992 INFO:teuthology.orchestra.run.trial027.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-13T02:19:32.992 INFO:teuthology.orchestra.run.trial027.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-13T02:19:32.993 INFO:teuthology.orchestra.run.trial027.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-13T02:19:32.993 INFO:teuthology.orchestra.run.trial027.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-13T02:19:32.993 INFO:teuthology.orchestra.run.trial027.stdout: openssl-devel-1:3.5.5-1.el9.x86_64 2026-02-13T02:19:32.993 INFO:teuthology.orchestra.run.trial027.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-13T02:19:32.993 INFO:teuthology.orchestra.run.trial027.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-13T02:19:32.993 INFO:teuthology.orchestra.run.trial027.stdout: perl-Thread-Queue-3.14-460.el9.noarch 2026-02-13T02:19:32.993 INFO:teuthology.orchestra.run.trial027.stdout: perl-threads-1:2.25-460.el9.x86_64 2026-02-13T02:19:32.993 INFO:teuthology.orchestra.run.trial027.stdout: perl-threads-shared-1.61-460.el9.x86_64 2026-02-13T02:19:32.993 INFO:teuthology.orchestra.run.trial027.stdout: postgresql-13.23-1.el9.x86_64 2026-02-13T02:19:32.993 INFO:teuthology.orchestra.run.trial027.stdout: postgresql-contrib-13.23-1.el9.x86_64 2026-02-13T02:19:32.993 INFO:teuthology.orchestra.run.trial027.stdout: postgresql-private-libs-13.23-1.el9.x86_64 2026-02-13T02:19:32.993 INFO:teuthology.orchestra.run.trial027.stdout: postgresql-server-13.23-1.el9.x86_64 2026-02-13T02:19:32.994 INFO:teuthology.orchestra.run.trial027.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-13T02:19:32.994 INFO:teuthology.orchestra.run.trial027.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-13T02:19:32.994 INFO:teuthology.orchestra.run.trial027.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-13T02:19:32.994 INFO:teuthology.orchestra.run.trial027.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-13T02:19:32.994 INFO:teuthology.orchestra.run.trial027.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-13T02:19:32.994 INFO:teuthology.orchestra.run.trial027.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-13T02:19:32.994 INFO:teuthology.orchestra.run.trial027.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-13T02:19:32.994 INFO:teuthology.orchestra.run.trial027.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-13T02:19:32.994 INFO:teuthology.orchestra.run.trial027.stdout: python3-ceph-argparse-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.994 INFO:teuthology.orchestra.run.trial027.stdout: python3-ceph-common-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.994 INFO:teuthology.orchestra.run.trial027.stdout: python3-cephfs-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.994 INFO:teuthology.orchestra.run.trial027.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-13T02:19:32.995 INFO:teuthology.orchestra.run.trial027.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-13T02:19:32.995 INFO:teuthology.orchestra.run.trial027.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-13T02:19:32.995 INFO:teuthology.orchestra.run.trial027.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-13T02:19:32.995 INFO:teuthology.orchestra.run.trial027.stdout: python3-cmd2-2.4.2-2.el9.noarch 2026-02-13T02:19:32.995 INFO:teuthology.orchestra.run.trial027.stdout: python3-colorama-0.4.6-3.el9.noarch 2026-02-13T02:19:32.995 INFO:teuthology.orchestra.run.trial027.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-13T02:19:32.995 INFO:teuthology.orchestra.run.trial027.stdout: python3-devel-3.9.25-2.el9.x86_64 2026-02-13T02:19:32.995 INFO:teuthology.orchestra.run.trial027.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-13T02:19:32.995 INFO:teuthology.orchestra.run.trial027.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-13T02:19:32.995 INFO:teuthology.orchestra.run.trial027.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-13T02:19:32.995 INFO:teuthology.orchestra.run.trial027.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-13T02:19:32.995 INFO:teuthology.orchestra.run.trial027.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-13T02:19:32.996 INFO:teuthology.orchestra.run.trial027.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-13T02:19:32.996 INFO:teuthology.orchestra.run.trial027.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-13T02:19:32.996 INFO:teuthology.orchestra.run.trial027.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-13T02:19:32.996 INFO:teuthology.orchestra.run.trial027.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-13T02:19:32.996 INFO:teuthology.orchestra.run.trial027.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-13T02:19:32.996 INFO:teuthology.orchestra.run.trial027.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-13T02:19:32.996 INFO:teuthology.orchestra.run.trial027.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-13T02:19:32.996 INFO:teuthology.orchestra.run.trial027.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-13T02:19:32.996 INFO:teuthology.orchestra.run.trial027.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-13T02:19:32.996 INFO:teuthology.orchestra.run.trial027.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-13T02:19:32.996 INFO:teuthology.orchestra.run.trial027.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-13T02:19:32.996 INFO:teuthology.orchestra.run.trial027.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-13T02:19:32.997 INFO:teuthology.orchestra.run.trial027.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-13T02:19:32.997 INFO:teuthology.orchestra.run.trial027.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-13T02:19:32.997 INFO:teuthology.orchestra.run.trial027.stdout: python3-ply-3.11-14.el9.noarch 2026-02-13T02:19:32.997 INFO:teuthology.orchestra.run.trial027.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-13T02:19:32.997 INFO:teuthology.orchestra.run.trial027.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-13T02:19:32.997 INFO:teuthology.orchestra.run.trial027.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-13T02:19:32.997 INFO:teuthology.orchestra.run.trial027.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-13T02:19:32.997 INFO:teuthology.orchestra.run.trial027.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-13T02:19:32.997 INFO:teuthology.orchestra.run.trial027.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-13T02:19:32.997 INFO:teuthology.orchestra.run.trial027.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2026-02-13T02:19:32.997 INFO:teuthology.orchestra.run.trial027.stdout: python3-rados-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.997 INFO:teuthology.orchestra.run.trial027.stdout: python3-rbd-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.997 INFO:teuthology.orchestra.run.trial027.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-13T02:19:32.998 INFO:teuthology.orchestra.run.trial027.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-13T02:19:32.998 INFO:teuthology.orchestra.run.trial027.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-13T02:19:32.998 INFO:teuthology.orchestra.run.trial027.stdout: python3-rgw-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.998 INFO:teuthology.orchestra.run.trial027.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-13T02:19:32.998 INFO:teuthology.orchestra.run.trial027.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-13T02:19:32.998 INFO:teuthology.orchestra.run.trial027.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-13T02:19:32.998 INFO:teuthology.orchestra.run.trial027.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-13T02:19:32.998 INFO:teuthology.orchestra.run.trial027.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-13T02:19:32.998 INFO:teuthology.orchestra.run.trial027.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-13T02:19:32.998 INFO:teuthology.orchestra.run.trial027.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-13T02:19:32.998 INFO:teuthology.orchestra.run.trial027.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2026-02-13T02:19:32.999 INFO:teuthology.orchestra.run.trial027.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-13T02:19:32.999 INFO:teuthology.orchestra.run.trial027.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-13T02:19:32.999 INFO:teuthology.orchestra.run.trial027.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-13T02:19:32.999 INFO:teuthology.orchestra.run.trial027.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-13T02:19:32.999 INFO:teuthology.orchestra.run.trial027.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-13T02:19:32.999 INFO:teuthology.orchestra.run.trial027.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-13T02:19:32.999 INFO:teuthology.orchestra.run.trial027.stdout: rbd-fuse-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.999 INFO:teuthology.orchestra.run.trial027.stdout: rbd-mirror-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.999 INFO:teuthology.orchestra.run.trial027.stdout: rbd-nbd-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:32.999 INFO:teuthology.orchestra.run.trial027.stdout: re2-1:20211101-20.el9.x86_64 2026-02-13T02:19:32.999 INFO:teuthology.orchestra.run.trial027.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-13T02:19:32.999 INFO:teuthology.orchestra.run.trial027.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-13T02:19:32.999 INFO:teuthology.orchestra.run.trial027.stdout: unzip-6.0-59.el9.x86_64 2026-02-13T02:19:33.000 INFO:teuthology.orchestra.run.trial027.stdout: uuid-1.6.2-55.el9.x86_64 2026-02-13T02:19:33.000 INFO:teuthology.orchestra.run.trial027.stdout: xfsprogs-devel-6.4.0-7.el9.x86_64 2026-02-13T02:19:33.000 INFO:teuthology.orchestra.run.trial027.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-13T02:19:33.000 INFO:teuthology.orchestra.run.trial027.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-13T02:19:33.000 INFO:teuthology.orchestra.run.trial027.stdout: zip-3.0-35.el9.x86_64 2026-02-13T02:19:33.000 INFO:teuthology.orchestra.run.trial027.stdout: zlib-devel-1.2.11-41.el9.x86_64 2026-02-13T02:19:33.000 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:19:33.000 INFO:teuthology.orchestra.run.trial027.stdout:Complete! 2026-02-13T02:19:33.089 DEBUG:teuthology.parallel:result is None 2026-02-13T02:19:33.394 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: openssl-libs-1:3.5.1-6.el9.x86_64 184/184 2026-02-13T02:19:33.394 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-2:20.3.0-5241.g8f8d3829.el9.x86_64 1/184 2026-02-13T02:19:33.395 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-base-2:20.3.0-5241.g8f8d3829.el9.x86_64 2/184 2026-02-13T02:19:33.395 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-common-2:20.3.0-5241.g8f8d3829.el9.x86_64 3/184 2026-02-13T02:19:33.395 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-fuse-2:20.3.0-5241.g8f8d3829.el9.x86_64 4/184 2026-02-13T02:19:33.395 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5241.g8f8d3 5/184 2026-02-13T02:19:33.395 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-mds-2:20.3.0-5241.g8f8d3829.el9.x86_64 6/184 2026-02-13T02:19:33.395 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-mgr-2:20.3.0-5241.g8f8d3829.el9.x86_64 7/184 2026-02-13T02:19:33.395 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-mon-2:20.3.0-5241.g8f8d3829.el9.x86_64 8/184 2026-02-13T02:19:33.395 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-osd-2:20.3.0-5241.g8f8d3829.el9.x86_64 9/184 2026-02-13T02:19:33.395 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-osd-classic-2:20.3.0-5241.g8f8d3829.el9.x86 10/184 2026-02-13T02:19:33.395 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5241.g8f8d3829.el9.x86 11/184 2026-02-13T02:19:33.395 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-radosgw-2:20.3.0-5241.g8f8d3829.el9.x86_64 12/184 2026-02-13T02:19:33.396 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-selinux-2:20.3.0-5241.g8f8d3829.el9.x86_64 13/184 2026-02-13T02:19:33.396 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-test-2:20.3.0-5241.g8f8d3829.el9.x86_64 14/184 2026-02-13T02:19:33.396 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : cephfs-mirror-2:20.3.0-5241.g8f8d3829.el9.x86_64 15/184 2026-02-13T02:19:33.396 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : cephfs-shell-2:20.3.0-5241.g8f8d3829.el9.x86_64 16/184 2026-02-13T02:19:33.396 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libcephfs-devel-2:20.3.0-5241.g8f8d3829.el9.x86_ 17/184 2026-02-13T02:19:33.396 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5241.g8f8d3829.el9.x86 18/184 2026-02-13T02:19:33.396 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libcephfs2-2:20.3.0-5241.g8f8d3829.el9.x86_64 19/184 2026-02-13T02:19:33.396 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libcephsqlite-2:20.3.0-5241.g8f8d3829.el9.x86_64 20/184 2026-02-13T02:19:33.396 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : librados-devel-2:20.3.0-5241.g8f8d3829.el9.x86_6 21/184 2026-02-13T02:19:33.396 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libradosstriper1-2:20.3.0-5241.g8f8d3829.el9.x86 22/184 2026-02-13T02:19:33.396 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : librgw2-2:20.3.0-5241.g8f8d3829.el9.x86_64 23/184 2026-02-13T02:19:33.396 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5241.g8f8d3829.el 24/184 2026-02-13T02:19:33.396 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-ceph-common-2:20.3.0-5241.g8f8d3829.el9. 25/184 2026-02-13T02:19:33.397 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-cephfs-2:20.3.0-5241.g8f8d3829.el9.x86_6 26/184 2026-02-13T02:19:33.397 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-rados-2:20.3.0-5241.g8f8d3829.el9.x86_64 27/184 2026-02-13T02:19:33.397 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-rbd-2:20.3.0-5241.g8f8d3829.el9.x86_64 28/184 2026-02-13T02:19:33.397 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-rgw-2:20.3.0-5241.g8f8d3829.el9.x86_64 29/184 2026-02-13T02:19:33.397 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : rbd-fuse-2:20.3.0-5241.g8f8d3829.el9.x86_64 30/184 2026-02-13T02:19:33.397 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : rbd-mirror-2:20.3.0-5241.g8f8d3829.el9.x86_64 31/184 2026-02-13T02:19:33.397 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : rbd-nbd-2:20.3.0-5241.g8f8d3829.el9.x86_64 32/184 2026-02-13T02:19:33.397 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5241.g8f8d3829. 33/184 2026-02-13T02:19:33.397 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5241.g8f8d3829.el9.noa 34/184 2026-02-13T02:19:33.397 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5241.g8f8d3829.el9.n 35/184 2026-02-13T02:19:33.397 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5241.g8f8 36/184 2026-02-13T02:19:33.397 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5241.g8f8d3829.el 37/184 2026-02-13T02:19:33.398 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5241.g8f8d3829.el9.noarch 38/184 2026-02-13T02:19:33.398 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5241.g8f8d3829.e 39/184 2026-02-13T02:19:33.398 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-volume-2:20.3.0-5241.g8f8d3829.el9.noarch 40/184 2026-02-13T02:19:33.398 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : cephadm-2:20.3.0-5241.g8f8d3829.el9.noarch 41/184 2026-02-13T02:19:33.398 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : cephfs-top-2:20.3.0-5241.g8f8d3829.el9.noarch 42/184 2026-02-13T02:19:33.398 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : cryptsetup-2.8.1-2.el9.x86_64 43/184 2026-02-13T02:19:33.398 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 44/184 2026-02-13T02:19:33.398 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 45/184 2026-02-13T02:19:33.398 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 46/184 2026-02-13T02:19:33.398 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 47/184 2026-02-13T02:19:33.398 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 48/184 2026-02-13T02:19:33.398 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 49/184 2026-02-13T02:19:33.398 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 50/184 2026-02-13T02:19:33.399 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 51/184 2026-02-13T02:19:33.399 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 52/184 2026-02-13T02:19:33.399 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 53/184 2026-02-13T02:19:33.399 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 54/184 2026-02-13T02:19:33.399 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-ply-3.11-14.el9.noarch 55/184 2026-02-13T02:19:33.399 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 56/184 2026-02-13T02:19:33.399 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 57/184 2026-02-13T02:19:33.399 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 58/184 2026-02-13T02:19:33.399 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : unzip-6.0-59.el9.x86_64 59/184 2026-02-13T02:19:33.399 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : zip-3.0-35.el9.x86_64 60/184 2026-02-13T02:19:33.399 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : automake-1.16.2-8.el9.noarch 61/184 2026-02-13T02:19:33.399 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : bison-3.7.4-5.el9.x86_64 62/184 2026-02-13T02:19:33.400 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 63/184 2026-02-13T02:19:33.400 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : elfutils-libelf-devel-0.194-1.el9.x86_64 64/184 2026-02-13T02:19:33.400 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : flex-2.6.4-9.el9.x86_64 65/184 2026-02-13T02:19:33.400 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 66/184 2026-02-13T02:19:33.400 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 67/184 2026-02-13T02:19:33.400 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 68/184 2026-02-13T02:19:33.400 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 69/184 2026-02-13T02:19:33.400 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 70/184 2026-02-13T02:19:33.400 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 71/184 2026-02-13T02:19:33.400 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 72/184 2026-02-13T02:19:33.400 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 73/184 2026-02-13T02:19:33.400 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 74/184 2026-02-13T02:19:33.400 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 75/184 2026-02-13T02:19:33.401 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libuuid-devel-2.37.4-21.el9.x86_64 76/184 2026-02-13T02:19:33.401 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 77/184 2026-02-13T02:19:33.401 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libzstd-devel-1.5.5-1.el9.x86_64 78/184 2026-02-13T02:19:33.401 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 79/184 2026-02-13T02:19:33.401 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : lua-5.4.4-4.el9.x86_64 80/184 2026-02-13T02:19:33.401 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 81/184 2026-02-13T02:19:33.401 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 82/184 2026-02-13T02:19:33.401 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : openssl-devel-1:3.5.5-1.el9.x86_64 83/184 2026-02-13T02:19:33.401 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : perl-Thread-Queue-3.14-460.el9.noarch 84/184 2026-02-13T02:19:33.401 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : perl-threads-1:2.25-460.el9.x86_64 85/184 2026-02-13T02:19:33.401 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : perl-threads-shared-1.61-460.el9.x86_64 86/184 2026-02-13T02:19:33.401 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : postgresql-13.23-1.el9.x86_64 87/184 2026-02-13T02:19:33.401 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : postgresql-contrib-13.23-1.el9.x86_64 88/184 2026-02-13T02:19:33.402 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : postgresql-private-libs-13.23-1.el9.x86_64 89/184 2026-02-13T02:19:33.402 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : postgresql-server-13.23-1.el9.x86_64 90/184 2026-02-13T02:19:33.402 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 91/184 2026-02-13T02:19:33.402 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 92/184 2026-02-13T02:19:33.402 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 93/184 2026-02-13T02:19:33.402 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-devel-3.9.25-2.el9.x86_64 94/184 2026-02-13T02:19:33.402 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 95/184 2026-02-13T02:19:33.402 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 96/184 2026-02-13T02:19:33.402 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 97/184 2026-02-13T02:19:33.402 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 98/184 2026-02-13T02:19:33.402 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 99/184 2026-02-13T02:19:33.402 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 100/184 2026-02-13T02:19:33.402 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 101/184 2026-02-13T02:19:33.403 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 102/184 2026-02-13T02:19:33.403 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 103/184 2026-02-13T02:19:33.403 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 104/184 2026-02-13T02:19:33.403 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 105/184 2026-02-13T02:19:33.403 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 106/184 2026-02-13T02:19:33.403 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 107/184 2026-02-13T02:19:33.403 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 108/184 2026-02-13T02:19:33.403 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 109/184 2026-02-13T02:19:33.403 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 110/184 2026-02-13T02:19:33.403 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 111/184 2026-02-13T02:19:33.403 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 112/184 2026-02-13T02:19:33.404 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 113/184 2026-02-13T02:19:33.404 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : xfsprogs-devel-6.4.0-7.el9.x86_64 114/184 2026-02-13T02:19:33.404 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 115/184 2026-02-13T02:19:33.404 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 116/184 2026-02-13T02:19:33.404 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 117/184 2026-02-13T02:19:33.404 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 118/184 2026-02-13T02:19:33.404 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 119/184 2026-02-13T02:19:33.404 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 120/184 2026-02-13T02:19:33.404 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 121/184 2026-02-13T02:19:33.404 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 122/184 2026-02-13T02:19:33.404 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 123/184 2026-02-13T02:19:33.405 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 124/184 2026-02-13T02:19:33.405 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 125/184 2026-02-13T02:19:33.405 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 126/184 2026-02-13T02:19:33.405 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 127/184 2026-02-13T02:19:33.405 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 128/184 2026-02-13T02:19:33.405 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 129/184 2026-02-13T02:19:33.405 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 130/184 2026-02-13T02:19:33.405 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 131/184 2026-02-13T02:19:33.405 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 132/184 2026-02-13T02:19:33.405 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 133/184 2026-02-13T02:19:33.405 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 134/184 2026-02-13T02:19:33.405 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 135/184 2026-02-13T02:19:33.406 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 136/184 2026-02-13T02:19:33.406 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 137/184 2026-02-13T02:19:33.406 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 138/184 2026-02-13T02:19:33.406 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 139/184 2026-02-13T02:19:33.406 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 140/184 2026-02-13T02:19:33.406 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 141/184 2026-02-13T02:19:33.406 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 142/184 2026-02-13T02:19:33.406 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 143/184 2026-02-13T02:19:33.406 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 144/184 2026-02-13T02:19:33.406 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 145/184 2026-02-13T02:19:33.406 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 146/184 2026-02-13T02:19:33.406 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 147/184 2026-02-13T02:19:33.407 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 148/184 2026-02-13T02:19:33.407 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 149/184 2026-02-13T02:19:33.407 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 150/184 2026-02-13T02:19:33.407 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 151/184 2026-02-13T02:19:33.407 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 152/184 2026-02-13T02:19:33.407 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 153/184 2026-02-13T02:19:33.407 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 154/184 2026-02-13T02:19:33.407 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 155/184 2026-02-13T02:19:33.407 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 156/184 2026-02-13T02:19:33.407 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 157/184 2026-02-13T02:19:33.407 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 158/184 2026-02-13T02:19:33.407 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 159/184 2026-02-13T02:19:33.408 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : re2-1:20211101-20.el9.x86_64 160/184 2026-02-13T02:19:33.408 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 161/184 2026-02-13T02:19:33.408 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 162/184 2026-02-13T02:19:33.408 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : librados2-2:20.3.0-5241.g8f8d3829.el9.x86_64 163/184 2026-02-13T02:19:33.408 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 164/184 2026-02-13T02:19:33.408 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : librbd1-2:20.3.0-5241.g8f8d3829.el9.x86_64 165/184 2026-02-13T02:19:33.408 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 166/184 2026-02-13T02:19:33.408 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : NetworkManager-1:1.54.3-2.el9.x86_64 167/184 2026-02-13T02:19:33.408 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : NetworkManager-1:1.54.1-1.el9.x86_64 168/184 2026-02-13T02:19:33.408 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : NetworkManager-libnm-1:1.54.3-2.el9.x86_64 169/184 2026-02-13T02:19:33.408 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : NetworkManager-libnm-1:1.54.1-1.el9.x86_64 170/184 2026-02-13T02:19:33.408 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : NetworkManager-team-1:1.54.3-2.el9.x86_64 171/184 2026-02-13T02:19:33.408 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : NetworkManager-team-1:1.54.1-1.el9.x86_64 172/184 2026-02-13T02:19:33.409 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : NetworkManager-tui-1:1.54.3-2.el9.x86_64 173/184 2026-02-13T02:19:33.409 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : NetworkManager-tui-1:1.54.1-1.el9.x86_64 174/184 2026-02-13T02:19:33.409 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : iproute-6.17.0-2.el9.x86_64 175/184 2026-02-13T02:19:33.409 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : iproute-6.17.0-1.el9.x86_64 176/184 2026-02-13T02:19:33.409 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : iproute-tc-6.17.0-2.el9.x86_64 177/184 2026-02-13T02:19:33.409 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : iproute-tc-6.17.0-1.el9.x86_64 178/184 2026-02-13T02:19:33.409 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : openssl-1:3.5.5-1.el9.x86_64 179/184 2026-02-13T02:19:33.409 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : openssl-1:3.5.1-6.el9.x86_64 180/184 2026-02-13T02:19:33.409 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : openssl-fips-provider-1:3.5.5-1.el9.x86_64 181/184 2026-02-13T02:19:33.409 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : openssl-fips-provider-1:3.5.1-6.el9.x86_64 182/184 2026-02-13T02:19:33.409 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : openssl-libs-1:3.5.5-1.el9.x86_64 183/184 2026-02-13T02:19:33.464 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : openssl-libs-1:3.5.1-6.el9.x86_64 184/184 2026-02-13T02:19:33.464 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:19:33.465 INFO:teuthology.orchestra.run.trial028.stdout:Upgraded: 2026-02-13T02:19:33.465 INFO:teuthology.orchestra.run.trial028.stdout: NetworkManager-1:1.54.3-2.el9.x86_64 2026-02-13T02:19:33.465 INFO:teuthology.orchestra.run.trial028.stdout: NetworkManager-libnm-1:1.54.3-2.el9.x86_64 2026-02-13T02:19:33.465 INFO:teuthology.orchestra.run.trial028.stdout: NetworkManager-team-1:1.54.3-2.el9.x86_64 2026-02-13T02:19:33.465 INFO:teuthology.orchestra.run.trial028.stdout: NetworkManager-tui-1:1.54.3-2.el9.x86_64 2026-02-13T02:19:33.465 INFO:teuthology.orchestra.run.trial028.stdout: iproute-6.17.0-2.el9.x86_64 2026-02-13T02:19:33.465 INFO:teuthology.orchestra.run.trial028.stdout: iproute-tc-6.17.0-2.el9.x86_64 2026-02-13T02:19:33.465 INFO:teuthology.orchestra.run.trial028.stdout: librados2-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.465 INFO:teuthology.orchestra.run.trial028.stdout: librbd1-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.465 INFO:teuthology.orchestra.run.trial028.stdout: openssl-1:3.5.5-1.el9.x86_64 2026-02-13T02:19:33.465 INFO:teuthology.orchestra.run.trial028.stdout: openssl-fips-provider-1:3.5.5-1.el9.x86_64 2026-02-13T02:19:33.466 INFO:teuthology.orchestra.run.trial028.stdout: openssl-libs-1:3.5.5-1.el9.x86_64 2026-02-13T02:19:33.466 INFO:teuthology.orchestra.run.trial028.stdout:Installed: 2026-02-13T02:19:33.466 INFO:teuthology.orchestra.run.trial028.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-13T02:19:33.466 INFO:teuthology.orchestra.run.trial028.stdout: automake-1.16.2-8.el9.noarch 2026-02-13T02:19:33.466 INFO:teuthology.orchestra.run.trial028.stdout: bison-3.7.4-5.el9.x86_64 2026-02-13T02:19:33.466 INFO:teuthology.orchestra.run.trial028.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-13T02:19:33.466 INFO:teuthology.orchestra.run.trial028.stdout: ceph-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.466 INFO:teuthology.orchestra.run.trial028.stdout: ceph-base-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.466 INFO:teuthology.orchestra.run.trial028.stdout: ceph-common-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.466 INFO:teuthology.orchestra.run.trial028.stdout: ceph-fuse-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.466 INFO:teuthology.orchestra.run.trial028.stdout: ceph-grafana-dashboards-2:20.3.0-5241.g8f8d3829.el9.noarch 2026-02-13T02:19:33.467 INFO:teuthology.orchestra.run.trial028.stdout: ceph-immutable-object-cache-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.467 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mds-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.467 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.467 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-cephadm-2:20.3.0-5241.g8f8d3829.el9.noarch 2026-02-13T02:19:33.467 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-dashboard-2:20.3.0-5241.g8f8d3829.el9.noarch 2026-02-13T02:19:33.467 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5241.g8f8d3829.el9.noarch 2026-02-13T02:19:33.467 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-modules-core-2:20.3.0-5241.g8f8d3829.el9.noarch 2026-02-13T02:19:33.467 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-rook-2:20.3.0-5241.g8f8d3829.el9.noarch 2026-02-13T02:19:33.467 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mon-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.467 INFO:teuthology.orchestra.run.trial028.stdout: ceph-osd-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.467 INFO:teuthology.orchestra.run.trial028.stdout: ceph-osd-classic-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.467 INFO:teuthology.orchestra.run.trial028.stdout: ceph-osd-crimson-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.467 INFO:teuthology.orchestra.run.trial028.stdout: ceph-prometheus-alerts-2:20.3.0-5241.g8f8d3829.el9.noarch 2026-02-13T02:19:33.468 INFO:teuthology.orchestra.run.trial028.stdout: ceph-radosgw-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.468 INFO:teuthology.orchestra.run.trial028.stdout: ceph-selinux-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.468 INFO:teuthology.orchestra.run.trial028.stdout: ceph-test-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.468 INFO:teuthology.orchestra.run.trial028.stdout: ceph-volume-2:20.3.0-5241.g8f8d3829.el9.noarch 2026-02-13T02:19:33.468 INFO:teuthology.orchestra.run.trial028.stdout: cephadm-2:20.3.0-5241.g8f8d3829.el9.noarch 2026-02-13T02:19:33.468 INFO:teuthology.orchestra.run.trial028.stdout: cephfs-mirror-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.468 INFO:teuthology.orchestra.run.trial028.stdout: cephfs-shell-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.468 INFO:teuthology.orchestra.run.trial028.stdout: cephfs-top-2:20.3.0-5241.g8f8d3829.el9.noarch 2026-02-13T02:19:33.468 INFO:teuthology.orchestra.run.trial028.stdout: cryptsetup-2.8.1-2.el9.x86_64 2026-02-13T02:19:33.468 INFO:teuthology.orchestra.run.trial028.stdout: elfutils-libelf-devel-0.194-1.el9.x86_64 2026-02-13T02:19:33.468 INFO:teuthology.orchestra.run.trial028.stdout: flex-2.6.4-9.el9.x86_64 2026-02-13T02:19:33.468 INFO:teuthology.orchestra.run.trial028.stdout: flexiblas-3.0.4-8.el9.x86_64 2026-02-13T02:19:33.469 INFO:teuthology.orchestra.run.trial028.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2026-02-13T02:19:33.469 INFO:teuthology.orchestra.run.trial028.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2026-02-13T02:19:33.469 INFO:teuthology.orchestra.run.trial028.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-13T02:19:33.469 INFO:teuthology.orchestra.run.trial028.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-13T02:19:33.469 INFO:teuthology.orchestra.run.trial028.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-13T02:19:33.469 INFO:teuthology.orchestra.run.trial028.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-13T02:19:33.469 INFO:teuthology.orchestra.run.trial028.stdout: libaio-devel-0.3.111-13.el9.x86_64 2026-02-13T02:19:33.469 INFO:teuthology.orchestra.run.trial028.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-13T02:19:33.469 INFO:teuthology.orchestra.run.trial028.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-13T02:19:33.469 INFO:teuthology.orchestra.run.trial028.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-13T02:19:33.469 INFO:teuthology.orchestra.run.trial028.stdout: libcephfs-devel-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.469 INFO:teuthology.orchestra.run.trial028.stdout: libcephfs-proxy2-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.470 INFO:teuthology.orchestra.run.trial028.stdout: libcephfs2-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.470 INFO:teuthology.orchestra.run.trial028.stdout: libcephsqlite-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.470 INFO:teuthology.orchestra.run.trial028.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-13T02:19:33.470 INFO:teuthology.orchestra.run.trial028.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-13T02:19:33.470 INFO:teuthology.orchestra.run.trial028.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-13T02:19:33.470 INFO:teuthology.orchestra.run.trial028.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-13T02:19:33.470 INFO:teuthology.orchestra.run.trial028.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-13T02:19:33.470 INFO:teuthology.orchestra.run.trial028.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-13T02:19:33.470 INFO:teuthology.orchestra.run.trial028.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-13T02:19:33.470 INFO:teuthology.orchestra.run.trial028.stdout: librados-devel-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.470 INFO:teuthology.orchestra.run.trial028.stdout: libradosstriper1-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.470 INFO:teuthology.orchestra.run.trial028.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-13T02:19:33.470 INFO:teuthology.orchestra.run.trial028.stdout: librgw2-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.471 INFO:teuthology.orchestra.run.trial028.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-13T02:19:33.471 INFO:teuthology.orchestra.run.trial028.stdout: libtool-2.4.6-46.el9.x86_64 2026-02-13T02:19:33.471 INFO:teuthology.orchestra.run.trial028.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-13T02:19:33.471 INFO:teuthology.orchestra.run.trial028.stdout: libuuid-devel-2.37.4-21.el9.x86_64 2026-02-13T02:19:33.471 INFO:teuthology.orchestra.run.trial028.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-13T02:19:33.471 INFO:teuthology.orchestra.run.trial028.stdout: libzstd-devel-1.5.5-1.el9.x86_64 2026-02-13T02:19:33.471 INFO:teuthology.orchestra.run.trial028.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-13T02:19:33.471 INFO:teuthology.orchestra.run.trial028.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-13T02:19:33.471 INFO:teuthology.orchestra.run.trial028.stdout: lua-5.4.4-4.el9.x86_64 2026-02-13T02:19:33.471 INFO:teuthology.orchestra.run.trial028.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-13T02:19:33.471 INFO:teuthology.orchestra.run.trial028.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-13T02:19:33.471 INFO:teuthology.orchestra.run.trial028.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-13T02:19:33.472 INFO:teuthology.orchestra.run.trial028.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-13T02:19:33.472 INFO:teuthology.orchestra.run.trial028.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-13T02:19:33.472 INFO:teuthology.orchestra.run.trial028.stdout: openssl-devel-1:3.5.5-1.el9.x86_64 2026-02-13T02:19:33.472 INFO:teuthology.orchestra.run.trial028.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-13T02:19:33.472 INFO:teuthology.orchestra.run.trial028.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-13T02:19:33.472 INFO:teuthology.orchestra.run.trial028.stdout: perl-Thread-Queue-3.14-460.el9.noarch 2026-02-13T02:19:33.472 INFO:teuthology.orchestra.run.trial028.stdout: perl-threads-1:2.25-460.el9.x86_64 2026-02-13T02:19:33.472 INFO:teuthology.orchestra.run.trial028.stdout: perl-threads-shared-1.61-460.el9.x86_64 2026-02-13T02:19:33.472 INFO:teuthology.orchestra.run.trial028.stdout: postgresql-13.23-1.el9.x86_64 2026-02-13T02:19:33.472 INFO:teuthology.orchestra.run.trial028.stdout: postgresql-contrib-13.23-1.el9.x86_64 2026-02-13T02:19:33.472 INFO:teuthology.orchestra.run.trial028.stdout: postgresql-private-libs-13.23-1.el9.x86_64 2026-02-13T02:19:33.472 INFO:teuthology.orchestra.run.trial028.stdout: postgresql-server-13.23-1.el9.x86_64 2026-02-13T02:19:33.472 INFO:teuthology.orchestra.run.trial028.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-13T02:19:33.473 INFO:teuthology.orchestra.run.trial028.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-13T02:19:33.473 INFO:teuthology.orchestra.run.trial028.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-13T02:19:33.473 INFO:teuthology.orchestra.run.trial028.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-13T02:19:33.473 INFO:teuthology.orchestra.run.trial028.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-13T02:19:33.473 INFO:teuthology.orchestra.run.trial028.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-13T02:19:33.473 INFO:teuthology.orchestra.run.trial028.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-13T02:19:33.473 INFO:teuthology.orchestra.run.trial028.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-13T02:19:33.473 INFO:teuthology.orchestra.run.trial028.stdout: python3-ceph-argparse-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.473 INFO:teuthology.orchestra.run.trial028.stdout: python3-ceph-common-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.473 INFO:teuthology.orchestra.run.trial028.stdout: python3-cephfs-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.473 INFO:teuthology.orchestra.run.trial028.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-13T02:19:33.474 INFO:teuthology.orchestra.run.trial028.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-13T02:19:33.474 INFO:teuthology.orchestra.run.trial028.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-13T02:19:33.474 INFO:teuthology.orchestra.run.trial028.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-13T02:19:33.474 INFO:teuthology.orchestra.run.trial028.stdout: python3-cmd2-2.4.2-2.el9.noarch 2026-02-13T02:19:33.474 INFO:teuthology.orchestra.run.trial028.stdout: python3-colorama-0.4.6-3.el9.noarch 2026-02-13T02:19:33.474 INFO:teuthology.orchestra.run.trial028.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-13T02:19:33.474 INFO:teuthology.orchestra.run.trial028.stdout: python3-devel-3.9.25-2.el9.x86_64 2026-02-13T02:19:33.474 INFO:teuthology.orchestra.run.trial028.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-13T02:19:33.474 INFO:teuthology.orchestra.run.trial028.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-13T02:19:33.474 INFO:teuthology.orchestra.run.trial028.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-13T02:19:33.474 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-13T02:19:33.474 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-13T02:19:33.474 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-13T02:19:33.475 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-13T02:19:33.475 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-13T02:19:33.475 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-13T02:19:33.475 INFO:teuthology.orchestra.run.trial028.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-13T02:19:33.475 INFO:teuthology.orchestra.run.trial028.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-13T02:19:33.475 INFO:teuthology.orchestra.run.trial028.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-13T02:19:33.475 INFO:teuthology.orchestra.run.trial028.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-13T02:19:33.475 INFO:teuthology.orchestra.run.trial028.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-13T02:19:33.475 INFO:teuthology.orchestra.run.trial028.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-13T02:19:33.475 INFO:teuthology.orchestra.run.trial028.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-13T02:19:33.475 INFO:teuthology.orchestra.run.trial028.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-13T02:19:33.475 INFO:teuthology.orchestra.run.trial028.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-13T02:19:33.476 INFO:teuthology.orchestra.run.trial028.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-13T02:19:33.476 INFO:teuthology.orchestra.run.trial028.stdout: python3-ply-3.11-14.el9.noarch 2026-02-13T02:19:33.476 INFO:teuthology.orchestra.run.trial028.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-13T02:19:33.476 INFO:teuthology.orchestra.run.trial028.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-13T02:19:33.476 INFO:teuthology.orchestra.run.trial028.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-13T02:19:33.476 INFO:teuthology.orchestra.run.trial028.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-13T02:19:33.476 INFO:teuthology.orchestra.run.trial028.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-13T02:19:33.476 INFO:teuthology.orchestra.run.trial028.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-13T02:19:33.476 INFO:teuthology.orchestra.run.trial028.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2026-02-13T02:19:33.476 INFO:teuthology.orchestra.run.trial028.stdout: python3-rados-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.476 INFO:teuthology.orchestra.run.trial028.stdout: python3-rbd-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.476 INFO:teuthology.orchestra.run.trial028.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-13T02:19:33.477 INFO:teuthology.orchestra.run.trial028.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-13T02:19:33.477 INFO:teuthology.orchestra.run.trial028.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-13T02:19:33.477 INFO:teuthology.orchestra.run.trial028.stdout: python3-rgw-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.477 INFO:teuthology.orchestra.run.trial028.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-13T02:19:33.477 INFO:teuthology.orchestra.run.trial028.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-13T02:19:33.477 INFO:teuthology.orchestra.run.trial028.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-13T02:19:33.477 INFO:teuthology.orchestra.run.trial028.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-13T02:19:33.477 INFO:teuthology.orchestra.run.trial028.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-13T02:19:33.477 INFO:teuthology.orchestra.run.trial028.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-13T02:19:33.477 INFO:teuthology.orchestra.run.trial028.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-13T02:19:33.477 INFO:teuthology.orchestra.run.trial028.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2026-02-13T02:19:33.477 INFO:teuthology.orchestra.run.trial028.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-13T02:19:33.477 INFO:teuthology.orchestra.run.trial028.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-13T02:19:33.478 INFO:teuthology.orchestra.run.trial028.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-13T02:19:33.478 INFO:teuthology.orchestra.run.trial028.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-13T02:19:33.478 INFO:teuthology.orchestra.run.trial028.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-13T02:19:33.478 INFO:teuthology.orchestra.run.trial028.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-13T02:19:33.478 INFO:teuthology.orchestra.run.trial028.stdout: rbd-fuse-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.478 INFO:teuthology.orchestra.run.trial028.stdout: rbd-mirror-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.478 INFO:teuthology.orchestra.run.trial028.stdout: rbd-nbd-2:20.3.0-5241.g8f8d3829.el9.x86_64 2026-02-13T02:19:33.478 INFO:teuthology.orchestra.run.trial028.stdout: re2-1:20211101-20.el9.x86_64 2026-02-13T02:19:33.478 INFO:teuthology.orchestra.run.trial028.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-13T02:19:33.478 INFO:teuthology.orchestra.run.trial028.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-13T02:19:33.478 INFO:teuthology.orchestra.run.trial028.stdout: unzip-6.0-59.el9.x86_64 2026-02-13T02:19:33.478 INFO:teuthology.orchestra.run.trial028.stdout: uuid-1.6.2-55.el9.x86_64 2026-02-13T02:19:33.478 INFO:teuthology.orchestra.run.trial028.stdout: xfsprogs-devel-6.4.0-7.el9.x86_64 2026-02-13T02:19:33.479 INFO:teuthology.orchestra.run.trial028.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-13T02:19:33.479 INFO:teuthology.orchestra.run.trial028.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-13T02:19:33.479 INFO:teuthology.orchestra.run.trial028.stdout: zip-3.0-35.el9.x86_64 2026-02-13T02:19:33.479 INFO:teuthology.orchestra.run.trial028.stdout: zlib-devel-1.2.11-41.el9.x86_64 2026-02-13T02:19:33.479 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:19:33.479 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-13T02:19:33.564 DEBUG:teuthology.parallel:result is None 2026-02-13T02:19:33.564 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f8d38294e4db8750b4b1649f62ce37caa63080f 2026-02-13T02:19:33.731 DEBUG:teuthology.orchestra.run.trial027:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-13T02:19:33.752 INFO:teuthology.orchestra.run.trial027.stdout:20.3.0-5241.g8f8d3829.el9 2026-02-13T02:19:33.753 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5241.g8f8d3829.el9 2026-02-13T02:19:33.753 INFO:teuthology.task.install:The correct ceph version 20.3.0-5241.g8f8d3829 is installed. 2026-02-13T02:19:33.756 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f8d38294e4db8750b4b1649f62ce37caa63080f 2026-02-13T02:19:33.914 DEBUG:teuthology.orchestra.run.trial028:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-13T02:19:33.934 INFO:teuthology.orchestra.run.trial028.stdout:20.3.0-5241.g8f8d3829.el9 2026-02-13T02:19:33.935 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5241.g8f8d3829.el9 2026-02-13T02:19:33.935 INFO:teuthology.task.install:The correct ceph version 20.3.0-5241.g8f8d3829 is installed. 2026-02-13T02:19:33.937 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f8d38294e4db8750b4b1649f62ce37caa63080f 2026-02-13T02:19:34.083 DEBUG:teuthology.orchestra.run.trial107:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-13T02:19:34.106 INFO:teuthology.orchestra.run.trial107.stdout:20.3.0-5241.g8f8d3829.el9 2026-02-13T02:19:34.106 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5241.g8f8d3829.el9 2026-02-13T02:19:34.106 INFO:teuthology.task.install:The correct ceph version 20.3.0-5241.g8f8d3829 is installed. 2026-02-13T02:19:34.108 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-13T02:19:34.108 DEBUG:teuthology.orchestra.run.trial027:> set -ex 2026-02-13T02:19:34.108 DEBUG:teuthology.orchestra.run.trial027:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-13T02:19:34.135 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-13T02:19:34.135 DEBUG:teuthology.orchestra.run.trial028:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-13T02:19:34.163 DEBUG:teuthology.orchestra.run.trial107:> set -ex 2026-02-13T02:19:34.164 DEBUG:teuthology.orchestra.run.trial107:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-13T02:19:34.192 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-13T02:19:34.193 DEBUG:teuthology.orchestra.run.trial027:> set -ex 2026-02-13T02:19:34.193 DEBUG:teuthology.orchestra.run.trial027:> sudo dd of=/usr/bin/daemon-helper 2026-02-13T02:19:34.217 DEBUG:teuthology.orchestra.run.trial027:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-13T02:19:34.281 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-13T02:19:34.281 DEBUG:teuthology.orchestra.run.trial028:> sudo dd of=/usr/bin/daemon-helper 2026-02-13T02:19:34.306 DEBUG:teuthology.orchestra.run.trial028:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-13T02:19:34.371 DEBUG:teuthology.orchestra.run.trial107:> set -ex 2026-02-13T02:19:34.371 DEBUG:teuthology.orchestra.run.trial107:> sudo dd of=/usr/bin/daemon-helper 2026-02-13T02:19:34.397 DEBUG:teuthology.orchestra.run.trial107:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-13T02:19:34.460 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-13T02:19:34.460 DEBUG:teuthology.orchestra.run.trial027:> set -ex 2026-02-13T02:19:34.460 DEBUG:teuthology.orchestra.run.trial027:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-13T02:19:34.484 DEBUG:teuthology.orchestra.run.trial027:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-13T02:19:34.549 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-13T02:19:34.549 DEBUG:teuthology.orchestra.run.trial028:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-13T02:19:34.575 DEBUG:teuthology.orchestra.run.trial028:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-13T02:19:34.640 DEBUG:teuthology.orchestra.run.trial107:> set -ex 2026-02-13T02:19:34.641 DEBUG:teuthology.orchestra.run.trial107:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-13T02:19:34.666 DEBUG:teuthology.orchestra.run.trial107:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-13T02:19:34.729 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-13T02:19:34.729 DEBUG:teuthology.orchestra.run.trial027:> set -ex 2026-02-13T02:19:34.729 DEBUG:teuthology.orchestra.run.trial027:> sudo dd of=/usr/bin/stdin-killer 2026-02-13T02:19:34.753 DEBUG:teuthology.orchestra.run.trial027:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-13T02:19:34.817 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-13T02:19:34.817 DEBUG:teuthology.orchestra.run.trial028:> sudo dd of=/usr/bin/stdin-killer 2026-02-13T02:19:34.842 DEBUG:teuthology.orchestra.run.trial028:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-13T02:19:34.907 DEBUG:teuthology.orchestra.run.trial107:> set -ex 2026-02-13T02:19:34.907 DEBUG:teuthology.orchestra.run.trial107:> sudo dd of=/usr/bin/stdin-killer 2026-02-13T02:19:34.932 DEBUG:teuthology.orchestra.run.trial107:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-13T02:19:34.996 INFO:teuthology.run_tasks:Running task cephadm... 2026-02-13T02:19:35.125 INFO:tasks.cephadm:Config: {'distro-suffix': 'rockylinux-10', 'roleless': False, 'skip_dashboard': True, 'cephfs': {'ec_profile': ['disabled'], 'max_mds': 1, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'client_fscrypt_dummy_encryption': True}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'is offline because no MDS', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'BLUESTORE_SLOW_OP_ALERT', 'slow operation indications in BlueStore', 'experiencing slow operations in BlueStore', 'MGR_MODULE_ERROR', 'OSD_DOWN', 'osd.* is down', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'osds down', 'osd.* is down', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'pg .* is stuck inactive', 'pg .* is .*degraded', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '8f8d38294e4db8750b4b1649f62ce37caa63080f', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2026-02-13T02:19:35.125 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 2026-02-13T02:19:35.128 INFO:tasks.cephadm:Cluster fsid is 707dbb29-0882-11f1-9262-d404e6e7d460 2026-02-13T02:19:35.128 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2026-02-13T02:19:35.128 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '10.20.193.27', 'mon.b': '10.20.193.28', 'mon.c': '10.20.193.107'} 2026-02-13T02:19:35.128 INFO:tasks.cephadm:First mon is mon.a on trial027 2026-02-13T02:19:35.128 INFO:tasks.cephadm:First mgr is x 2026-02-13T02:19:35.128 INFO:tasks.cephadm:Normalizing hostnames... 2026-02-13T02:19:35.128 DEBUG:teuthology.orchestra.run.trial027:> sudo hostname $(hostname -s) 2026-02-13T02:19:35.154 DEBUG:teuthology.orchestra.run.trial028:> sudo hostname $(hostname -s) 2026-02-13T02:19:35.178 DEBUG:teuthology.orchestra.run.trial107:> sudo hostname $(hostname -s) 2026-02-13T02:19:35.203 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2026-02-13T02:19:35.203 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f8d38294e4db8750b4b1649f62ce37caa63080f 2026-02-13T02:19:35.340 INFO:tasks.cephadm:builder_project result: [{'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20260210.171210/8f8d38294e4db8750b4b1649f62ce37caa63080f/centos/9/flavors/default/', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20260210.171210/8f8d38294e4db8750b4b1649f62ce37caa63080f/centos/9/flavors/default/', 'ref': 'wip-pdonnell-testing-20260210.171210', 'sha1': '8f8d38294e4db8750b4b1649f62ce37caa63080f', 'distro': 'centos', 'distro_version': '9', 'distro_codename': None, 'modified': '2026-02-10 22:03:44.386901', 'status': 'ready', 'flavor': 'default', 'project': 'ceph', 'archs': ['source', 'x86_64'], 'extra': {'version': '20.3.0-5241-g8f8d3829', 'package_manager_version': '20.3.0-5241.g8f8d3829', 'build_url': 'https://jenkins.ceph.com/job/ceph-dev-pipeline/2911/', 'root_build_cause': '', 'node_name': '10.20.192.26+soko16', 'job_name': 'ceph-dev-pipeline'}}] 2026-02-13T02:19:35.476 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-pdonnell-testing-20260210.171210, sha1 8f8d38294e4db8750b4b1649f62ce37caa63080f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=8f8d38294e4db8750b4b1649f62ce37caa63080f 2026-02-13T02:19:35.478 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20260210.171210/8f8d38294e4db8750b4b1649f62ce37caa63080f/centos/9/x86_64/flavors/default/cephadm 2026-02-13T02:19:35.478 INFO:tasks.cephadm:Downloading cephadm from url: https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20260210.171210/8f8d38294e4db8750b4b1649f62ce37caa63080f/centos/9/x86_64/flavors/default/cephadm 2026-02-13T02:19:35.478 DEBUG:teuthology.orchestra.run.trial027:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20260210.171210/8f8d38294e4db8750b4b1649f62ce37caa63080f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2026-02-13T02:19:35.809 INFO:teuthology.orchestra.run.trial027.stdout:-rw-r--r--. 1 ubuntu ubuntu 1073479 Feb 13 02:19 /home/ubuntu/cephtest/cephadm 2026-02-13T02:19:35.809 DEBUG:teuthology.orchestra.run.trial028:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20260210.171210/8f8d38294e4db8750b4b1649f62ce37caa63080f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2026-02-13T02:19:36.126 INFO:teuthology.orchestra.run.trial028.stdout:-rw-r--r--. 1 ubuntu ubuntu 1073479 Feb 13 02:19 /home/ubuntu/cephtest/cephadm 2026-02-13T02:19:36.126 DEBUG:teuthology.orchestra.run.trial107:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20260210.171210/8f8d38294e4db8750b4b1649f62ce37caa63080f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2026-02-13T02:19:36.427 INFO:teuthology.orchestra.run.trial107.stdout:-rw-r--r--. 1 ubuntu ubuntu 1073479 Feb 13 02:19 /home/ubuntu/cephtest/cephadm 2026-02-13T02:19:36.428 DEBUG:teuthology.orchestra.run.trial027:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2026-02-13T02:19:36.447 DEBUG:teuthology.orchestra.run.trial028:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2026-02-13T02:19:36.466 DEBUG:teuthology.orchestra.run.trial107:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2026-02-13T02:19:36.499 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 on all hosts... 2026-02-13T02:19:36.499 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 pull 2026-02-13T02:19:36.502 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 pull 2026-02-13T02:19:36.508 DEBUG:teuthology.orchestra.run.trial107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 pull 2026-02-13T02:19:36.662 INFO:teuthology.orchestra.run.trial028.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10... 2026-02-13T02:19:36.676 INFO:teuthology.orchestra.run.trial027.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10... 2026-02-13T02:19:36.677 INFO:teuthology.orchestra.run.trial107.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10... 2026-02-13T02:19:47.282 INFO:teuthology.orchestra.run.trial027.stdout:{ 2026-02-13T02:19:47.282 INFO:teuthology.orchestra.run.trial027.stdout: "ceph_version": "ceph version 20.3.0-5241-g8f8d3829 (8f8d38294e4db8750b4b1649f62ce37caa63080f) tentacle (dev)", 2026-02-13T02:19:47.282 INFO:teuthology.orchestra.run.trial027.stdout: "image_id": "b86a4a15c2dcd78db08124ee64cb48fcd681d6db032e313da19d063813b235da", 2026-02-13T02:19:47.282 INFO:teuthology.orchestra.run.trial027.stdout: "repo_digests": [ 2026-02-13T02:19:47.282 INFO:teuthology.orchestra.run.trial027.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9" 2026-02-13T02:19:47.282 INFO:teuthology.orchestra.run.trial027.stdout: ] 2026-02-13T02:19:47.282 INFO:teuthology.orchestra.run.trial027.stdout:} 2026-02-13T02:19:47.306 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-13T02:19:47.306 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_version": "ceph version 20.3.0-5241-g8f8d3829 (8f8d38294e4db8750b4b1649f62ce37caa63080f) tentacle (dev)", 2026-02-13T02:19:47.306 INFO:teuthology.orchestra.run.trial028.stdout: "image_id": "b86a4a15c2dcd78db08124ee64cb48fcd681d6db032e313da19d063813b235da", 2026-02-13T02:19:47.306 INFO:teuthology.orchestra.run.trial028.stdout: "repo_digests": [ 2026-02-13T02:19:47.306 INFO:teuthology.orchestra.run.trial028.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9" 2026-02-13T02:19:47.306 INFO:teuthology.orchestra.run.trial028.stdout: ] 2026-02-13T02:19:47.306 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-13T02:19:47.928 INFO:teuthology.orchestra.run.trial107.stdout:{ 2026-02-13T02:19:47.929 INFO:teuthology.orchestra.run.trial107.stdout: "ceph_version": "ceph version 20.3.0-5241-g8f8d3829 (8f8d38294e4db8750b4b1649f62ce37caa63080f) tentacle (dev)", 2026-02-13T02:19:47.929 INFO:teuthology.orchestra.run.trial107.stdout: "image_id": "b86a4a15c2dcd78db08124ee64cb48fcd681d6db032e313da19d063813b235da", 2026-02-13T02:19:47.929 INFO:teuthology.orchestra.run.trial107.stdout: "repo_digests": [ 2026-02-13T02:19:47.929 INFO:teuthology.orchestra.run.trial107.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9" 2026-02-13T02:19:47.929 INFO:teuthology.orchestra.run.trial107.stdout: ] 2026-02-13T02:19:47.929 INFO:teuthology.orchestra.run.trial107.stdout:} 2026-02-13T02:19:47.945 DEBUG:teuthology.orchestra.run.trial027:> sudo mkdir -p /etc/ceph 2026-02-13T02:19:47.975 DEBUG:teuthology.orchestra.run.trial028:> sudo mkdir -p /etc/ceph 2026-02-13T02:19:48.005 DEBUG:teuthology.orchestra.run.trial107:> sudo mkdir -p /etc/ceph 2026-02-13T02:19:48.038 DEBUG:teuthology.orchestra.run.trial027:> sudo chmod 777 /etc/ceph 2026-02-13T02:19:48.064 DEBUG:teuthology.orchestra.run.trial028:> sudo chmod 777 /etc/ceph 2026-02-13T02:19:48.089 DEBUG:teuthology.orchestra.run.trial107:> sudo chmod 777 /etc/ceph 2026-02-13T02:19:48.112 INFO:tasks.cephadm:Writing seed config... 2026-02-13T02:19:48.112 INFO:tasks.cephadm: override: [client] client_fscrypt_dummy_encryption = True 2026-02-13T02:19:48.112 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2026-02-13T02:19:48.112 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2026-02-13T02:19:48.113 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2026-02-13T02:19:48.113 INFO:tasks.cephadm: override: [mon] debug mon = 20 2026-02-13T02:19:48.113 INFO:tasks.cephadm: override: [mon] debug ms = 1 2026-02-13T02:19:48.113 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2026-02-13T02:19:48.113 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2026-02-13T02:19:48.113 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2026-02-13T02:19:48.113 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2026-02-13T02:19:48.113 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2026-02-13T02:19:48.113 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2026-02-13T02:19:48.113 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2026-02-13T02:19:48.113 INFO:tasks.cephadm: override: [osd] debug ms = 1 2026-02-13T02:19:48.113 INFO:tasks.cephadm: override: [osd] debug osd = 20 2026-02-13T02:19:48.113 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2026-02-13T02:19:48.113 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2026-02-13T02:19:48.113 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2026-02-13T02:19:48.113 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2026-02-13T02:19:48.113 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2026-02-13T02:19:48.113 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2026-02-13T02:19:48.113 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2026-02-13T02:19:48.113 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2026-02-13T02:19:48.113 DEBUG:teuthology.orchestra.run.trial027:> set -ex 2026-02-13T02:19:48.113 DEBUG:teuthology.orchestra.run.trial027:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2026-02-13T02:19:48.129 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 707dbb29-0882-11f1-9262-d404e6e7d460 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd mclock skip benchmark = true osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client_fscrypt_dummy_encryption = True 2026-02-13T02:19:48.129 DEBUG:teuthology.orchestra.run.trial027:mon.a> sudo journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mon.a.service 2026-02-13T02:19:48.171 DEBUG:teuthology.orchestra.run.trial027:mgr.x> sudo journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mgr.x.service 2026-02-13T02:19:48.213 INFO:tasks.cephadm:Bootstrapping... 2026-02-13T02:19:48.213 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 -v bootstrap --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 10.20.193.27 --skip-dashboard --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2026-02-13T02:19:48.352 INFO:teuthology.orchestra.run.trial027.stdout:-------------------------------------------------------------------------------- 2026-02-13T02:19:48.352 INFO:teuthology.orchestra.run.trial027.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10', '-v', 'bootstrap', '--fsid', '707dbb29-0882-11f1-9262-d404e6e7d460', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '10.20.193.27', '--skip-dashboard', '--skip-admin-label'] 2026-02-13T02:19:48.352 INFO:teuthology.orchestra.run.trial027.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2026-02-13T02:19:48.352 INFO:teuthology.orchestra.run.trial027.stdout:Verifying podman|docker is present... 2026-02-13T02:19:48.368 INFO:teuthology.orchestra.run.trial027.stdout:/bin/podman: stdout 5.6.0 2026-02-13T02:19:48.368 INFO:teuthology.orchestra.run.trial027.stdout:Verifying lvm2 is present... 2026-02-13T02:19:48.368 INFO:teuthology.orchestra.run.trial027.stdout:Verifying time synchronization is in place... 2026-02-13T02:19:48.375 INFO:teuthology.orchestra.run.trial027.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2026-02-13T02:19:48.375 INFO:teuthology.orchestra.run.trial027.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2026-02-13T02:19:48.381 INFO:teuthology.orchestra.run.trial027.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2026-02-13T02:19:48.381 INFO:teuthology.orchestra.run.trial027.stdout:systemctl: stdout inactive 2026-02-13T02:19:48.387 INFO:teuthology.orchestra.run.trial027.stdout:systemctl: stdout enabled 2026-02-13T02:19:48.392 INFO:teuthology.orchestra.run.trial027.stdout:systemctl: stdout active 2026-02-13T02:19:48.393 INFO:teuthology.orchestra.run.trial027.stdout:Unit chronyd.service is enabled and running 2026-02-13T02:19:48.393 INFO:teuthology.orchestra.run.trial027.stdout:Repeating the final host check... 2026-02-13T02:19:48.409 INFO:teuthology.orchestra.run.trial027.stdout:/bin/podman: stdout 5.6.0 2026-02-13T02:19:48.409 INFO:teuthology.orchestra.run.trial027.stdout:podman (/bin/podman) version 5.6.0 is present 2026-02-13T02:19:48.409 INFO:teuthology.orchestra.run.trial027.stdout:systemctl is present 2026-02-13T02:19:48.409 INFO:teuthology.orchestra.run.trial027.stdout:lvcreate is present 2026-02-13T02:19:48.415 INFO:teuthology.orchestra.run.trial027.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2026-02-13T02:19:48.415 INFO:teuthology.orchestra.run.trial027.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2026-02-13T02:19:48.421 INFO:teuthology.orchestra.run.trial027.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2026-02-13T02:19:48.421 INFO:teuthology.orchestra.run.trial027.stdout:systemctl: stdout inactive 2026-02-13T02:19:48.427 INFO:teuthology.orchestra.run.trial027.stdout:systemctl: stdout enabled 2026-02-13T02:19:48.432 INFO:teuthology.orchestra.run.trial027.stdout:systemctl: stdout active 2026-02-13T02:19:48.432 INFO:teuthology.orchestra.run.trial027.stdout:Unit chronyd.service is enabled and running 2026-02-13T02:19:48.432 INFO:teuthology.orchestra.run.trial027.stdout:Host looks OK 2026-02-13T02:19:48.433 INFO:teuthology.orchestra.run.trial027.stdout:Cluster fsid: 707dbb29-0882-11f1-9262-d404e6e7d460 2026-02-13T02:19:48.433 INFO:teuthology.orchestra.run.trial027.stdout:Acquiring lock 140134847946224 on /run/cephadm/707dbb29-0882-11f1-9262-d404e6e7d460.lock 2026-02-13T02:19:48.433 INFO:teuthology.orchestra.run.trial027.stdout:Lock 140134847946224 acquired on /run/cephadm/707dbb29-0882-11f1-9262-d404e6e7d460.lock 2026-02-13T02:19:48.433 INFO:teuthology.orchestra.run.trial027.stdout:Verifying IP 10.20.193.27 port 3300 ... 2026-02-13T02:19:48.433 INFO:teuthology.orchestra.run.trial027.stdout:Verifying IP 10.20.193.27 port 6789 ... 2026-02-13T02:19:48.433 INFO:teuthology.orchestra.run.trial027.stdout:Base mon IP(s) is [10.20.193.27:3300, 10.20.193.27:6789], mon addrv is [v2:10.20.193.27:3300,v1:10.20.193.27:6789] 2026-02-13T02:19:48.436 INFO:teuthology.orchestra.run.trial027.stdout:/sbin/ip: stdout default via 10.20.192.1 dev enp1s0f0 proto dhcp src 10.20.193.27 metric 102 2026-02-13T02:19:48.437 INFO:teuthology.orchestra.run.trial027.stdout:/sbin/ip: stdout 10.20.192.0/20 dev enp1s0f0 proto kernel scope link src 10.20.193.27 metric 102 2026-02-13T02:19:48.437 INFO:teuthology.orchestra.run.trial027.stdout:/sbin/ip: stdout 169.254.3.0/24 dev enp7s0f4u2u2c2 proto kernel scope link src 169.254.3.1 metric 101 2026-02-13T02:19:48.439 INFO:teuthology.orchestra.run.trial027.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2026-02-13T02:19:48.439 INFO:teuthology.orchestra.run.trial027.stdout:/sbin/ip: stdout fe80::/64 dev enp1s0f0 proto kernel metric 256 pref medium 2026-02-13T02:19:48.439 INFO:teuthology.orchestra.run.trial027.stdout:/sbin/ip: stdout fe80::/64 dev enp7s0f4u2u2c2 proto kernel metric 1024 pref medium 2026-02-13T02:19:48.442 INFO:teuthology.orchestra.run.trial027.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2026-02-13T02:19:48.442 INFO:teuthology.orchestra.run.trial027.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2026-02-13T02:19:48.442 INFO:teuthology.orchestra.run.trial027.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2026-02-13T02:19:48.442 INFO:teuthology.orchestra.run.trial027.stdout:/sbin/ip: stdout 2: enp7s0f4u2u2c2: mtu 1500 state UNKNOWN qlen 1000 2026-02-13T02:19:48.442 INFO:teuthology.orchestra.run.trial027.stdout:/sbin/ip: stdout inet6 fe80::39be:77d1:5124:5691/64 scope link noprefixroute 2026-02-13T02:19:48.442 INFO:teuthology.orchestra.run.trial027.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2026-02-13T02:19:48.442 INFO:teuthology.orchestra.run.trial027.stdout:/sbin/ip: stdout 3: enp1s0f0: mtu 1500 state UP qlen 1000 2026-02-13T02:19:48.442 INFO:teuthology.orchestra.run.trial027.stdout:/sbin/ip: stdout inet6 fe80::925a:8ff:fe77:6260/64 scope link 2026-02-13T02:19:48.442 INFO:teuthology.orchestra.run.trial027.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2026-02-13T02:19:48.442 INFO:teuthology.orchestra.run.trial027.stdout:Mon IP `10.20.193.27` is in CIDR network `10.20.192.0/20` 2026-02-13T02:19:48.442 INFO:teuthology.orchestra.run.trial027.stdout:Mon IP `10.20.193.27` is in CIDR network `10.20.192.0/20` 2026-02-13T02:19:48.442 INFO:teuthology.orchestra.run.trial027.stdout:Inferred mon public CIDR from local network configuration ['10.20.192.0/20', '10.20.192.0/20'] 2026-02-13T02:19:48.443 INFO:teuthology.orchestra.run.trial027.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2026-02-13T02:19:48.443 INFO:teuthology.orchestra.run.trial027.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10... 2026-02-13T02:19:48.680 INFO:teuthology.orchestra.run.trial027.stdout:/bin/podman: stdout b86a4a15c2dcd78db08124ee64cb48fcd681d6db032e313da19d063813b235da 2026-02-13T02:19:48.680 INFO:teuthology.orchestra.run.trial027.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10... 2026-02-13T02:19:48.680 INFO:teuthology.orchestra.run.trial027.stdout:/bin/podman: stderr Getting image source signatures 2026-02-13T02:19:48.680 INFO:teuthology.orchestra.run.trial027.stdout:/bin/podman: stderr Copying blob sha256:9b7325f35ad40fc8a795fce82955bc486ff370c187eaa160f4df65d0fb22f2a2 2026-02-13T02:19:48.680 INFO:teuthology.orchestra.run.trial027.stdout:/bin/podman: stderr Copying blob sha256:0c53e9fad0324c188a40a9b1e0ca4eb777da2812507ec104055fa6ec400d9a27 2026-02-13T02:19:48.680 INFO:teuthology.orchestra.run.trial027.stdout:/bin/podman: stderr Copying config sha256:b86a4a15c2dcd78db08124ee64cb48fcd681d6db032e313da19d063813b235da 2026-02-13T02:19:48.680 INFO:teuthology.orchestra.run.trial027.stdout:/bin/podman: stderr Writing manifest to image destination 2026-02-13T02:19:48.816 INFO:teuthology.orchestra.run.trial027.stdout:ceph: stdout ceph version 20.3.0-5241-g8f8d3829 (8f8d38294e4db8750b4b1649f62ce37caa63080f) tentacle (dev) 2026-02-13T02:19:48.816 INFO:teuthology.orchestra.run.trial027.stdout:Ceph version: ceph version 20.3.0-5241-g8f8d3829 (8f8d38294e4db8750b4b1649f62ce37caa63080f) tentacle (dev) 2026-02-13T02:19:48.816 INFO:teuthology.orchestra.run.trial027.stdout:Extracting ceph user uid/gid from container image... 2026-02-13T02:19:48.975 INFO:teuthology.orchestra.run.trial027.stdout:stat: stdout 167 167 2026-02-13T02:19:48.975 INFO:teuthology.orchestra.run.trial027.stdout:Creating initial keys... 2026-02-13T02:19:49.093 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph-authtool: stdout AQBFio5pWPngAxAA4WHrltNoq/8phTCG7Q/P8Q== 2026-02-13T02:19:49.231 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph-authtool: stdout AQBFio5pKKgwDBAAsknyDXpLQn6S11KNbKWFRg== 2026-02-13T02:19:49.354 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph-authtool: stdout AQBFio5pfmqqExAA8OVtErcEExuJtr46CYhEjQ== 2026-02-13T02:19:49.355 INFO:teuthology.orchestra.run.trial027.stdout:Creating initial monmap... 2026-02-13T02:19:49.488 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2026-02-13T02:19:49.488 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = tentacle 2026-02-13T02:19:49.488 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 707dbb29-0882-11f1-9262-d404e6e7d460 2026-02-13T02:19:49.488 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2026-02-13T02:19:49.488 INFO:teuthology.orchestra.run.trial027.stdout:monmaptool for a [v2:10.20.193.27:3300,v1:10.20.193.27:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2026-02-13T02:19:49.488 INFO:teuthology.orchestra.run.trial027.stdout:setting min_mon_release = tentacle 2026-02-13T02:19:49.488 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/monmaptool: set fsid to 707dbb29-0882-11f1-9262-d404e6e7d460 2026-02-13T02:19:49.488 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2026-02-13T02:19:49.488 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:19:49.488 INFO:teuthology.orchestra.run.trial027.stdout:Creating mon... 2026-02-13T02:19:49.608 INFO:teuthology.orchestra.run.trial027.stdout:create mon.a on 2026-02-13T02:19:49.780 INFO:teuthology.orchestra.run.trial027.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2026-02-13T02:19:49.885 INFO:teuthology.orchestra.run.trial027.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2026-02-13T02:19:50.006 INFO:teuthology.orchestra.run.trial027.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-707dbb29-0882-11f1-9262-d404e6e7d460.target → /etc/systemd/system/ceph-707dbb29-0882-11f1-9262-d404e6e7d460.target. 2026-02-13T02:19:50.006 INFO:teuthology.orchestra.run.trial027.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-707dbb29-0882-11f1-9262-d404e6e7d460.target → /etc/systemd/system/ceph-707dbb29-0882-11f1-9262-d404e6e7d460.target. 2026-02-13T02:19:50.141 INFO:teuthology.orchestra.run.trial027.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mon.a 2026-02-13T02:19:50.141 INFO:teuthology.orchestra.run.trial027.stdout:systemctl: stderr Failed to reset failed state of unit ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mon.a.service: Unit ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mon.a.service not loaded. 2026-02-13T02:19:50.263 INFO:teuthology.orchestra.run.trial027.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-707dbb29-0882-11f1-9262-d404e6e7d460.target.wants/ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mon.a.service → /etc/systemd/system/ceph-707dbb29-0882-11f1-9262-d404e6e7d460@.service. 2026-02-13T02:19:50.397 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:50 trial027 podman[24102]: 2026-02-13 02:19:50.396909247 +0000 UTC m=+0.046270511 container start bff7d9e8f172ab3360f5fbdb0740d65021412813809964b3cb9f0be5585b190f (image=quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mon-a, org.opencontainers.image.licenses=BSD-3-Clause, OSD_FLAVOR=default, io.buildah.version=1.33.7, version=10, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, summary=Rocky Linux Base image, org.opencontainers.image.version=10, name=rockylinux, ceph=True, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, org.opencontainers.image.title=rockylinux, CEPH_REF=wip-pdonnell-testing-20260210.171210, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, license=BSD-3-Clause, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git) 2026-02-13T02:19:50.411 INFO:teuthology.orchestra.run.trial027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2026-02-13T02:19:50.411 INFO:teuthology.orchestra.run.trial027.stdout:systemctl: stdout disabled 2026-02-13T02:19:50.417 INFO:teuthology.orchestra.run.trial027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2026-02-13T02:19:50.418 INFO:teuthology.orchestra.run.trial027.stdout:systemctl: stdout inactive 2026-02-13T02:19:50.418 INFO:teuthology.orchestra.run.trial027.stdout:firewalld.service is not enabled 2026-02-13T02:19:50.418 INFO:teuthology.orchestra.run.trial027.stdout:Not possible to enable service . firewalld.service is not available 2026-02-13T02:19:50.418 INFO:teuthology.orchestra.run.trial027.stdout:Waiting for mon to start... 2026-02-13T02:19:50.418 INFO:teuthology.orchestra.run.trial027.stdout:Waiting for mon... 2026-02-13T02:19:50.579 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout cluster: 2026-02-13T02:19:50.580 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout id: 707dbb29-0882-11f1-9262-d404e6e7d460 2026-02-13T02:19:50.580 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2026-02-13T02:19:50.580 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout 2026-02-13T02:19:50.580 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout services: 2026-02-13T02:19:50.580 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.138619s) [leader: a] 2026-02-13T02:19:50.580 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2026-02-13T02:19:50.580 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2026-02-13T02:19:50.580 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout 2026-02-13T02:19:50.580 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout data: 2026-02-13T02:19:50.580 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2026-02-13T02:19:50.580 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2026-02-13T02:19:50.581 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2026-02-13T02:19:50.581 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout pgs: 2026-02-13T02:19:50.581 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout 2026-02-13T02:19:50.581 INFO:teuthology.orchestra.run.trial027.stdout:mon is available 2026-02-13T02:19:50.581 INFO:teuthology.orchestra.run.trial027.stdout:Assimilating anything we can from ceph.conf... 2026-02-13T02:19:50.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:50 trial027 bash[24102]: bff7d9e8f172ab3360f5fbdb0740d65021412813809964b3cb9f0be5585b190f 2026-02-13T02:19:50.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:50 trial027 podman[24102]: 2026-02-13 02:19:50.360062873 +0000 UTC m=+0.009424137 image pull b86a4a15c2dcd78db08124ee64cb48fcd681d6db032e313da19d063813b235da quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 2026-02-13T02:19:50.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:50 trial027 systemd[1]: Started Ceph mon.a for 707dbb29-0882-11f1-9262-d404e6e7d460. 2026-02-13T02:19:50.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:50 trial027 ceph-mon[24129]: mkfs 707dbb29-0882-11f1-9262-d404e6e7d460 2026-02-13T02:19:50.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:50 trial027 ceph-mon[24129]: mon.a is new leader, mons a in quorum (ranks 0) 2026-02-13T02:19:50.739 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout 2026-02-13T02:19:50.739 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout [global] 2026-02-13T02:19:50.739 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout fsid = 707dbb29-0882-11f1-9262-d404e6e7d460 2026-02-13T02:19:50.739 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2026-02-13T02:19:50.739 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout mon_host = [v2:10.20.193.27:3300,v1:10.20.193.27:6789] 2026-02-13T02:19:50.739 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2026-02-13T02:19:50.739 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2026-02-13T02:19:50.739 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2026-02-13T02:19:50.740 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2026-02-13T02:19:50.740 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout 2026-02-13T02:19:50.740 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout [mgr] 2026-02-13T02:19:50.740 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2026-02-13T02:19:50.740 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout 2026-02-13T02:19:50.740 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout [osd] 2026-02-13T02:19:50.740 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2026-02-13T02:19:50.740 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2026-02-13T02:19:50.740 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2026-02-13T02:19:50.740 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2026-02-13T02:19:50.740 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2026-02-13T02:19:50.740 INFO:teuthology.orchestra.run.trial027.stdout:Generating new minimal ceph.conf... 2026-02-13T02:19:50.892 INFO:teuthology.orchestra.run.trial027.stdout:Restarting the monitor... 2026-02-13T02:19:50.968 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:50 trial027 systemd[1]: Stopping Ceph mon.a for 707dbb29-0882-11f1-9262-d404e6e7d460... 2026-02-13T02:19:51.142 INFO:teuthology.orchestra.run.trial027.stdout:Setting public_network to 10.20.192.0/20 in global config section 2026-02-13T02:19:51.261 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:50 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mon-a[24125]: 2026-02-13T02:19:50.965+0000 7fc3e269e6c0 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2026-02-13T02:19:51.261 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:50 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mon-a[24125]: 2026-02-13T02:19:50.965+0000 7fc3e269e6c0 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2026-02-13T02:19:51.261 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:50 trial027 podman[24410]: 2026-02-13 02:19:50.976947447 +0000 UTC m=+0.022131880 container died bff7d9e8f172ab3360f5fbdb0740d65021412813809964b3cb9f0be5585b190f (image=quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mon-a, org.opencontainers.image.title=rockylinux, CEPH_REF=wip-pdonnell-testing-20260210.171210, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, vendor=Rocky Enterprise Software Foundation, summary=Rocky Linux Base image, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.version=10, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, license=BSD-3-Clause, org.opencontainers.image.licenses=BSD-3-Clause, io.buildah.version=1.33.7, OSD_FLAVOR=default, ceph=True, name=rockylinux, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, version=10, org.opencontainers.image.authors=Ceph Release Team ) 2026-02-13T02:19:51.261 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:50 trial027 podman[24410]: 2026-02-13 02:19:50.985470999 +0000 UTC m=+0.030655432 container remove bff7d9e8f172ab3360f5fbdb0740d65021412813809964b3cb9f0be5585b190f (image=quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mon-a, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, vendor=Rocky Enterprise Software Foundation, license=BSD-3-Clause, org.opencontainers.image.licenses=BSD-3-Clause, version=10, org.opencontainers.image.title=rockylinux, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, summary=Rocky Linux Base image, CEPH_REF=wip-pdonnell-testing-20260210.171210, ceph=True, org.opencontainers.image.version=10, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, OSD_FLAVOR=default, name=rockylinux, org.opencontainers.image.authors=Ceph Release Team , io.buildah.version=1.33.7) 2026-02-13T02:19:51.262 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:50 trial027 bash[24410]: ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mon-a 2026-02-13T02:19:51.262 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 systemd[1]: ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mon.a.service: Deactivated successfully. 2026-02-13T02:19:51.262 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 systemd[1]: Stopped Ceph mon.a for 707dbb29-0882-11f1-9262-d404e6e7d460. 2026-02-13T02:19:51.262 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 systemd[1]: Starting Ceph mon.a for 707dbb29-0882-11f1-9262-d404e6e7d460... 2026-02-13T02:19:51.262 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 podman[24536]: 2026-02-13 02:19:51.110065995 +0000 UTC m=+0.015844919 container create 5749214d12b0f2f4fd1fde8eb0871adb9e56451e20504ba9a4a69dc2e9e4e251 (image=quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mon-a, vendor=Rocky Enterprise Software Foundation, CEPH_REF=wip-pdonnell-testing-20260210.171210, org.opencontainers.image.documentation=https://docs.ceph.com/, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, summary=Rocky Linux Base image, org.opencontainers.image.licenses=BSD-3-Clause, io.buildah.version=1.33.7, OSD_FLAVOR=default, org.opencontainers.image.authors=Ceph Release Team , license=BSD-3-Clause, org.opencontainers.image.version=10, version=10, name=rockylinux, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.title=rockylinux, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, ceph=True, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10) 2026-02-13T02:19:51.262 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 podman[24536]: 2026-02-13 02:19:51.134959697 +0000 UTC m=+0.040738611 container init 5749214d12b0f2f4fd1fde8eb0871adb9e56451e20504ba9a4a69dc2e9e4e251 (image=quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mon-a, OSD_FLAVOR=default, vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, org.opencontainers.image.licenses=BSD-3-Clause, summary=Rocky Linux Base image, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, CEPH_REF=wip-pdonnell-testing-20260210.171210, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, ceph=True, org.opencontainers.image.authors=Ceph Release Team , GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.title=rockylinux, license=BSD-3-Clause, org.opencontainers.image.version=10, version=10, name=rockylinux, io.buildah.version=1.33.7) 2026-02-13T02:19:51.262 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 podman[24536]: 2026-02-13 02:19:51.137583231 +0000 UTC m=+0.043362155 container start 5749214d12b0f2f4fd1fde8eb0871adb9e56451e20504ba9a4a69dc2e9e4e251 (image=quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mon-a, license=BSD-3-Clause, version=10, name=rockylinux, io.buildah.version=1.33.7, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.version=10, vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, org.opencontainers.image.authors=Ceph Release Team , ceph=True, summary=Rocky Linux Base image, org.opencontainers.image.title=rockylinux, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, CEPH_REF=wip-pdonnell-testing-20260210.171210, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, org.opencontainers.image.licenses=BSD-3-Clause, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, OSD_FLAVOR=default) 2026-02-13T02:19:51.262 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 bash[24536]: 5749214d12b0f2f4fd1fde8eb0871adb9e56451e20504ba9a4a69dc2e9e4e251 2026-02-13T02:19:51.262 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 podman[24536]: 2026-02-13 02:19:51.103940315 +0000 UTC m=+0.009719249 image pull b86a4a15c2dcd78db08124ee64cb48fcd681d6db032e313da19d063813b235da quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 2026-02-13T02:19:51.262 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 systemd[1]: Started Ceph mon.a for 707dbb29-0882-11f1-9262-d404e6e7d460. 2026-02-13T02:19:51.262 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: set uid:gid to 167:167 (ceph:ceph) 2026-02-13T02:19:51.263 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: ceph version 20.3.0-5241-g8f8d3829 (8f8d38294e4db8750b4b1649f62ce37caa63080f) tentacle (dev - RelWithDebInfo), process ceph-mon, pid 2 2026-02-13T02:19:51.263 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: pidfile_write: ignore empty --pid-file 2026-02-13T02:19:51.263 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: load: jerasure load: lrc 2026-02-13T02:19:51.263 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: RocksDB version: 7.9.2 2026-02-13T02:19:51.263 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Git sha 0 2026-02-13T02:19:51.263 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Compile date 2026-02-10 21:31:46 2026-02-13T02:19:51.263 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: DB SUMMARY 2026-02-13T02:19:51.263 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: DB Session ID: CC3GQPO1HLT8OJI6L18X 2026-02-13T02:19:51.263 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: CURRENT file: CURRENT 2026-02-13T02:19:51.263 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: IDENTITY file: IDENTITY 2026-02-13T02:19:51.263 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2026-02-13T02:19:51.263 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2026-02-13T02:19:51.264 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 82213 ; 2026-02-13T02:19:51.264 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.error_if_exists: 0 2026-02-13T02:19:51.264 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.create_if_missing: 0 2026-02-13T02:19:51.264 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.paranoid_checks: 1 2026-02-13T02:19:51.264 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.flush_verify_memtable_count: 1 2026-02-13T02:19:51.264 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2026-02-13T02:19:51.264 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2026-02-13T02:19:51.264 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.env: 0x55b1439742c0 2026-02-13T02:19:51.264 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.fs: PosixFileSystem 2026-02-13T02:19:51.264 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.info_log: 0x55b14471cd60 2026-02-13T02:19:51.264 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.max_file_opening_threads: 16 2026-02-13T02:19:51.264 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.statistics: (nil) 2026-02-13T02:19:51.264 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.use_fsync: 0 2026-02-13T02:19:51.265 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.max_log_file_size: 0 2026-02-13T02:19:51.265 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.max_manifest_file_size: 1073741824 2026-02-13T02:19:51.265 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.log_file_time_to_roll: 0 2026-02-13T02:19:51.265 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.keep_log_file_num: 1000 2026-02-13T02:19:51.265 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.recycle_log_file_num: 0 2026-02-13T02:19:51.265 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.allow_fallocate: 1 2026-02-13T02:19:51.265 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.allow_mmap_reads: 0 2026-02-13T02:19:51.265 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.allow_mmap_writes: 0 2026-02-13T02:19:51.265 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.use_direct_reads: 0 2026-02-13T02:19:51.265 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2026-02-13T02:19:51.265 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.create_missing_column_families: 0 2026-02-13T02:19:51.265 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.db_log_dir: 2026-02-13T02:19:51.265 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.wal_dir: 2026-02-13T02:19:51.266 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.table_cache_numshardbits: 6 2026-02-13T02:19:51.266 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.WAL_ttl_seconds: 0 2026-02-13T02:19:51.266 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.WAL_size_limit_MB: 0 2026-02-13T02:19:51.266 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2026-02-13T02:19:51.266 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.manifest_preallocation_size: 4194304 2026-02-13T02:19:51.266 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.is_fd_close_on_exec: 1 2026-02-13T02:19:51.266 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.advise_random_on_open: 1 2026-02-13T02:19:51.266 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.db_write_buffer_size: 0 2026-02-13T02:19:51.266 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.write_buffer_manager: 0x55b144713b80 2026-02-13T02:19:51.266 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.access_hint_on_compaction_start: 1 2026-02-13T02:19:51.266 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.random_access_max_buffer_size: 1048576 2026-02-13T02:19:51.266 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.use_adaptive_mutex: 0 2026-02-13T02:19:51.266 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.rate_limiter: (nil) 2026-02-13T02:19:51.267 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2026-02-13T02:19:51.267 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.wal_recovery_mode: 2 2026-02-13T02:19:51.267 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.enable_thread_tracking: 0 2026-02-13T02:19:51.267 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.enable_pipelined_write: 0 2026-02-13T02:19:51.267 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.unordered_write: 0 2026-02-13T02:19:51.267 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.allow_concurrent_memtable_write: 1 2026-02-13T02:19:51.267 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2026-02-13T02:19:51.267 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.write_thread_max_yield_usec: 100 2026-02-13T02:19:51.267 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.write_thread_slow_yield_usec: 3 2026-02-13T02:19:51.267 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.row_cache: None 2026-02-13T02:19:51.267 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.wal_filter: None 2026-02-13T02:19:51.267 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.avoid_flush_during_recovery: 0 2026-02-13T02:19:51.268 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.allow_ingest_behind: 0 2026-02-13T02:19:51.268 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.two_write_queues: 0 2026-02-13T02:19:51.268 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.manual_wal_flush: 0 2026-02-13T02:19:51.268 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.wal_compression: 0 2026-02-13T02:19:51.268 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.atomic_flush: 0 2026-02-13T02:19:51.268 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2026-02-13T02:19:51.268 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.persist_stats_to_disk: 0 2026-02-13T02:19:51.268 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.write_dbid_to_manifest: 0 2026-02-13T02:19:51.268 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.log_readahead_size: 0 2026-02-13T02:19:51.268 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.file_checksum_gen_factory: Unknown 2026-02-13T02:19:51.268 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.best_efforts_recovery: 0 2026-02-13T02:19:51.268 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2026-02-13T02:19:51.268 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2026-02-13T02:19:51.269 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.allow_data_in_errors: 0 2026-02-13T02:19:51.269 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.db_host_id: __hostname__ 2026-02-13T02:19:51.269 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.enforce_single_del_contracts: true 2026-02-13T02:19:51.269 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.max_background_jobs: 2 2026-02-13T02:19:51.269 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.max_background_compactions: -1 2026-02-13T02:19:51.269 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.max_subcompactions: 1 2026-02-13T02:19:51.269 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.avoid_flush_during_shutdown: 0 2026-02-13T02:19:51.269 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2026-02-13T02:19:51.269 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.delayed_write_rate : 16777216 2026-02-13T02:19:51.269 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.max_total_wal_size: 0 2026-02-13T02:19:51.269 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2026-02-13T02:19:51.269 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.stats_dump_period_sec: 600 2026-02-13T02:19:51.270 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.stats_persist_period_sec: 600 2026-02-13T02:19:51.270 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.stats_history_buffer_size: 1048576 2026-02-13T02:19:51.270 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.max_open_files: -1 2026-02-13T02:19:51.270 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.bytes_per_sync: 0 2026-02-13T02:19:51.270 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.wal_bytes_per_sync: 0 2026-02-13T02:19:51.270 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.strict_bytes_per_sync: 0 2026-02-13T02:19:51.270 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.compaction_readahead_size: 0 2026-02-13T02:19:51.270 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.max_background_flushes: -1 2026-02-13T02:19:51.270 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Compression algorithms supported: 2026-02-13T02:19:51.270 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: kZSTD supported: 0 2026-02-13T02:19:51.270 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: kXpressCompression supported: 0 2026-02-13T02:19:51.270 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: kBZip2Compression supported: 0 2026-02-13T02:19:51.270 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: kZSTDNotFinalCompression supported: 0 2026-02-13T02:19:51.271 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: kLZ4Compression supported: 1 2026-02-13T02:19:51.271 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: kZlibCompression supported: 1 2026-02-13T02:19:51.271 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: kLZ4HCCompression supported: 1 2026-02-13T02:19:51.271 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: kSnappyCompression supported: 1 2026-02-13T02:19:51.271 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Fast CRC32 supported: Supported on x86 2026-02-13T02:19:51.271 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: DMutex implementation: pthread_mutex_t 2026-02-13T02:19:51.271 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2026-02-13T02:19:51.271 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2026-02-13T02:19:51.271 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-13T02:19:51.271 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.merge_operator: 2026-02-13T02:19:51.271 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.compaction_filter: None 2026-02-13T02:19:51.271 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.compaction_filter_factory: None 2026-02-13T02:19:51.271 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.sst_partitioner_factory: None 2026-02-13T02:19:51.272 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.memtable_factory: SkipListFactory 2026-02-13T02:19:51.272 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.table_factory: BlockBasedTable 2026-02-13T02:19:51.272 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b14489c360) 2026-02-13T02:19:51.272 INFO:journalctl@ceph.mon.a.trial027.stdout: cache_index_and_filter_blocks: 1 2026-02-13T02:19:51.272 INFO:journalctl@ceph.mon.a.trial027.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2026-02-13T02:19:51.272 INFO:journalctl@ceph.mon.a.trial027.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2026-02-13T02:19:51.272 INFO:journalctl@ceph.mon.a.trial027.stdout: pin_top_level_index_and_filter: 1 2026-02-13T02:19:51.272 INFO:journalctl@ceph.mon.a.trial027.stdout: index_type: 0 2026-02-13T02:19:51.272 INFO:journalctl@ceph.mon.a.trial027.stdout: data_block_index_type: 0 2026-02-13T02:19:51.272 INFO:journalctl@ceph.mon.a.trial027.stdout: index_shortening: 1 2026-02-13T02:19:51.272 INFO:journalctl@ceph.mon.a.trial027.stdout: data_block_hash_table_util_ratio: 0.750000 2026-02-13T02:19:51.272 INFO:journalctl@ceph.mon.a.trial027.stdout: checksum: 4 2026-02-13T02:19:51.272 INFO:journalctl@ceph.mon.a.trial027.stdout: no_block_cache: 0 2026-02-13T02:19:51.273 INFO:journalctl@ceph.mon.a.trial027.stdout: block_cache: 0x55b144715690 2026-02-13T02:19:51.273 INFO:journalctl@ceph.mon.a.trial027.stdout: block_cache_name: BinnedLRUCache 2026-02-13T02:19:51.273 INFO:journalctl@ceph.mon.a.trial027.stdout: block_cache_options: 2026-02-13T02:19:51.273 INFO:journalctl@ceph.mon.a.trial027.stdout: capacity : 536870912 2026-02-13T02:19:51.273 INFO:journalctl@ceph.mon.a.trial027.stdout: num_shard_bits : 4 2026-02-13T02:19:51.273 INFO:journalctl@ceph.mon.a.trial027.stdout: strict_capacity_limit : 0 2026-02-13T02:19:51.273 INFO:journalctl@ceph.mon.a.trial027.stdout: high_pri_pool_ratio: 0.000 2026-02-13T02:19:51.273 INFO:journalctl@ceph.mon.a.trial027.stdout: block_cache_compressed: (nil) 2026-02-13T02:19:51.273 INFO:journalctl@ceph.mon.a.trial027.stdout: persistent_cache: (nil) 2026-02-13T02:19:51.273 INFO:journalctl@ceph.mon.a.trial027.stdout: block_size: 4096 2026-02-13T02:19:51.273 INFO:journalctl@ceph.mon.a.trial027.stdout: block_size_deviation: 10 2026-02-13T02:19:51.273 INFO:journalctl@ceph.mon.a.trial027.stdout: block_restart_interval: 16 2026-02-13T02:19:51.273 INFO:journalctl@ceph.mon.a.trial027.stdout: index_block_restart_interval: 1 2026-02-13T02:19:51.274 INFO:journalctl@ceph.mon.a.trial027.stdout: metadata_block_size: 4096 2026-02-13T02:19:51.274 INFO:journalctl@ceph.mon.a.trial027.stdout: partition_filters: 0 2026-02-13T02:19:51.274 INFO:journalctl@ceph.mon.a.trial027.stdout: use_delta_encoding: 1 2026-02-13T02:19:51.274 INFO:journalctl@ceph.mon.a.trial027.stdout: filter_policy: bloomfilter 2026-02-13T02:19:51.274 INFO:journalctl@ceph.mon.a.trial027.stdout: whole_key_filtering: 1 2026-02-13T02:19:51.274 INFO:journalctl@ceph.mon.a.trial027.stdout: verify_compression: 0 2026-02-13T02:19:51.274 INFO:journalctl@ceph.mon.a.trial027.stdout: read_amp_bytes_per_bit: 0 2026-02-13T02:19:51.274 INFO:journalctl@ceph.mon.a.trial027.stdout: format_version: 5 2026-02-13T02:19:51.274 INFO:journalctl@ceph.mon.a.trial027.stdout: enable_index_compression: 1 2026-02-13T02:19:51.274 INFO:journalctl@ceph.mon.a.trial027.stdout: block_align: 0 2026-02-13T02:19:51.274 INFO:journalctl@ceph.mon.a.trial027.stdout: max_auto_readahead_size: 262144 2026-02-13T02:19:51.274 INFO:journalctl@ceph.mon.a.trial027.stdout: prepopulate_block_cache: 0 2026-02-13T02:19:51.274 INFO:journalctl@ceph.mon.a.trial027.stdout: initial_auto_readahead_size: 8192 2026-02-13T02:19:51.275 INFO:journalctl@ceph.mon.a.trial027.stdout: num_file_reads_for_auto_readahead: 2 2026-02-13T02:19:51.275 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.write_buffer_size: 33554432 2026-02-13T02:19:51.275 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.max_write_buffer_number: 2 2026-02-13T02:19:51.275 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.compression: NoCompression 2026-02-13T02:19:51.275 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.bottommost_compression: Disabled 2026-02-13T02:19:51.275 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.prefix_extractor: nullptr 2026-02-13T02:19:51.275 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-13T02:19:51.275 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.num_levels: 7 2026-02-13T02:19:51.275 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2026-02-13T02:19:51.275 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-13T02:19:51.275 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-13T02:19:51.275 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-13T02:19:51.275 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-13T02:19:51.276 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-13T02:19:51.276 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-13T02:19:51.276 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-13T02:19:51.276 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-13T02:19:51.276 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-13T02:19:51.276 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-13T02:19:51.276 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-13T02:19:51.276 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.compression_opts.window_bits: -14 2026-02-13T02:19:51.276 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.compression_opts.level: 32767 2026-02-13T02:19:51.276 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.compression_opts.strategy: 0 2026-02-13T02:19:51.276 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-13T02:19:51.276 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-13T02:19:51.276 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-13T02:19:51.277 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-13T02:19:51.277 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.compression_opts.enabled: false 2026-02-13T02:19:51.277 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-13T02:19:51.277 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2026-02-13T02:19:51.277 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-13T02:19:51.277 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-13T02:19:51.277 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.target_file_size_base: 67108864 2026-02-13T02:19:51.277 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.target_file_size_multiplier: 1 2026-02-13T02:19:51.277 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.max_bytes_for_level_base: 268435456 2026-02-13T02:19:51.277 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2026-02-13T02:19:51.277 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2026-02-13T02:19:51.277 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-13T02:19:51.277 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-13T02:19:51.278 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-13T02:19:51.278 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-13T02:19:51.278 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-13T02:19:51.278 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-13T02:19:51.278 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-13T02:19:51.278 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-13T02:19:51.278 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-13T02:19:51.278 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-13T02:19:51.278 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.arena_block_size: 1048576 2026-02-13T02:19:51.278 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-13T02:19:51.278 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-13T02:19:51.278 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.disable_auto_compactions: 0 2026-02-13T02:19:51.278 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-13T02:19:51.279 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-13T02:19:51.279 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-13T02:19:51.279 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-13T02:19:51.279 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-13T02:19:51.279 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-13T02:19:51.279 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-13T02:19:51.279 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-13T02:19:51.279 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-13T02:19:51.279 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-13T02:19:51.279 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-13T02:19:51.279 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.inplace_update_support: 0 2026-02-13T02:19:51.279 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.inplace_update_num_locks: 10000 2026-02-13T02:19:51.279 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-13T02:19:51.280 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-13T02:19:51.280 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.memtable_huge_page_size: 0 2026-02-13T02:19:51.280 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.bloom_locality: 0 2026-02-13T02:19:51.280 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.max_successive_merges: 0 2026-02-13T02:19:51.280 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.optimize_filters_for_hits: 0 2026-02-13T02:19:51.280 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.paranoid_file_checks: 0 2026-02-13T02:19:51.280 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.force_consistency_checks: 1 2026-02-13T02:19:51.280 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.report_bg_io_stats: 0 2026-02-13T02:19:51.280 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.ttl: 2592000 2026-02-13T02:19:51.280 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.periodic_compaction_seconds: 0 2026-02-13T02:19:51.280 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-13T02:19:51.280 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-13T02:19:51.280 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.enable_blob_files: false 2026-02-13T02:19:51.281 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.min_blob_size: 0 2026-02-13T02:19:51.281 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.blob_file_size: 268435456 2026-02-13T02:19:51.281 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.blob_compression_type: NoCompression 2026-02-13T02:19:51.281 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.enable_blob_garbage_collection: false 2026-02-13T02:19:51.281 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-13T02:19:51.281 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-13T02:19:51.281 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-13T02:19:51.281 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.blob_file_starting_level: 0 2026-02-13T02:19:51.281 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-13T02:19:51.281 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2026-02-13T02:19:51.281 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2026-02-13T02:19:51.281 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 473e5dca-d296-423f-8017-1b058bcccaad 2026-02-13T02:19:51.281 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: EVENT_LOG_v1 {"time_micros": 1770949191153341, "job": 1, "event": "recovery_started", "wal_files": [9]} 2026-02-13T02:19:51.282 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2026-02-13T02:19:51.282 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: EVENT_LOG_v1 {"time_micros": 1770949191154278, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 78864, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 251, "table_properties": {"data_size": 77041, "index_size": 208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 11114, "raw_average_key_size": 50, "raw_value_size": 70773, "raw_average_value_size": 318, "num_data_blocks": 9, "num_entries": 222, "num_filter_entries": 222, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1770949191, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "473e5dca-d296-423f-8017-1b058bcccaad", "db_session_id": "CC3GQPO1HLT8OJI6L18X", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2026-02-13T02:19:51.282 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: EVENT_LOG_v1 {"time_micros": 1770949191154352, "job": 1, "event": "recovery_finished"} 2026-02-13T02:19:51.282 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2026-02-13T02:19:51.282 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2026-02-13T02:19:51.282 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b144738e00 2026-02-13T02:19:51.282 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: DB pointer 0x55b14553e000 2026-02-13T02:19:51.282 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2026-02-13T02:19:51.282 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: rocksdb: [db/db_impl/db_impl.cc:1111] 2026-02-13T02:19:51.282 INFO:journalctl@ceph.mon.a.trial027.stdout: ** DB Stats ** 2026-02-13T02:19:51.282 INFO:journalctl@ceph.mon.a.trial027.stdout: Uptime(secs): 0.0 total, 0.0 interval 2026-02-13T02:19:51.283 INFO:journalctl@ceph.mon.a.trial027.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2026-02-13T02:19:51.283 INFO:journalctl@ceph.mon.a.trial027.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2026-02-13T02:19:51.283 INFO:journalctl@ceph.mon.a.trial027.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2026-02-13T02:19:51.283 INFO:journalctl@ceph.mon.a.trial027.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2026-02-13T02:19:51.283 INFO:journalctl@ceph.mon.a.trial027.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2026-02-13T02:19:51.283 INFO:journalctl@ceph.mon.a.trial027.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2026-02-13T02:19:51.283 INFO:journalctl@ceph.mon.a.trial027.stdout: 2026-02-13T02:19:51.283 INFO:journalctl@ceph.mon.a.trial027.stdout: ** Compaction Stats [default] ** 2026-02-13T02:19:51.283 INFO:journalctl@ceph.mon.a.trial027.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2026-02-13T02:19:51.283 INFO:journalctl@ceph.mon.a.trial027.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2026-02-13T02:19:51.283 INFO:journalctl@ceph.mon.a.trial027.stdout: L0 2/0 78.88 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 102.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2026-02-13T02:19:51.283 INFO:journalctl@ceph.mon.a.trial027.stdout: Sum 2/0 78.88 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 102.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2026-02-13T02:19:51.284 INFO:journalctl@ceph.mon.a.trial027.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 102.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2026-02-13T02:19:51.284 INFO:journalctl@ceph.mon.a.trial027.stdout: 2026-02-13T02:19:51.284 INFO:journalctl@ceph.mon.a.trial027.stdout: ** Compaction Stats [default] ** 2026-02-13T02:19:51.284 INFO:journalctl@ceph.mon.a.trial027.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2026-02-13T02:19:51.284 INFO:journalctl@ceph.mon.a.trial027.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2026-02-13T02:19:51.284 INFO:journalctl@ceph.mon.a.trial027.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 102.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2026-02-13T02:19:51.284 INFO:journalctl@ceph.mon.a.trial027.stdout: 2026-02-13T02:19:51.284 INFO:journalctl@ceph.mon.a.trial027.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2026-02-13T02:19:51.284 INFO:journalctl@ceph.mon.a.trial027.stdout: 2026-02-13T02:19:51.284 INFO:journalctl@ceph.mon.a.trial027.stdout: Uptime(secs): 0.0 total, 0.0 interval 2026-02-13T02:19:51.284 INFO:journalctl@ceph.mon.a.trial027.stdout: Flush(GB): cumulative 0.000, interval 0.000 2026-02-13T02:19:51.284 INFO:journalctl@ceph.mon.a.trial027.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2026-02-13T02:19:51.285 INFO:journalctl@ceph.mon.a.trial027.stdout: AddFile(Total Files): cumulative 0, interval 0 2026-02-13T02:19:51.285 INFO:journalctl@ceph.mon.a.trial027.stdout: AddFile(L0 Files): cumulative 0, interval 0 2026-02-13T02:19:51.285 INFO:journalctl@ceph.mon.a.trial027.stdout: AddFile(Keys): cumulative 0, interval 0 2026-02-13T02:19:51.285 INFO:journalctl@ceph.mon.a.trial027.stdout: Cumulative compaction: 0.00 GB write, 22.94 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2026-02-13T02:19:51.285 INFO:journalctl@ceph.mon.a.trial027.stdout: Interval compaction: 0.00 GB write, 22.94 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2026-02-13T02:19:51.285 INFO:journalctl@ceph.mon.a.trial027.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2026-02-13T02:19:51.285 INFO:journalctl@ceph.mon.a.trial027.stdout: Block cache BinnedLRUCache@0x55b144715690#2 capacity: 512.00 MB usage: 1.16 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 6e-06 secs_since: 0 2026-02-13T02:19:51.285 INFO:journalctl@ceph.mon.a.trial027.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.39 KB,7.45058e-05%) Misc(1,0.00 KB,0%) 2026-02-13T02:19:51.285 INFO:journalctl@ceph.mon.a.trial027.stdout: 2026-02-13T02:19:51.285 INFO:journalctl@ceph.mon.a.trial027.stdout: ** File Read Latency Histogram By Level [default] ** 2026-02-13T02:19:51.285 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: starting mon.a rank 0 at public addrs [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] at bind addrs [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 707dbb29-0882-11f1-9262-d404e6e7d460 2026-02-13T02:19:51.285 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: mon.a@-1(???) e1 preinit fsid 707dbb29-0882-11f1-9262-d404e6e7d460 2026-02-13T02:19:51.286 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: mon.a@-1(???).mds e1 new map 2026-02-13T02:19:51.286 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: mon.a@-1(???).mds e1 print_map 2026-02-13T02:19:51.286 INFO:journalctl@ceph.mon.a.trial027.stdout: e1 2026-02-13T02:19:51.286 INFO:journalctl@ceph.mon.a.trial027.stdout: btime 2026-02-13T02:19:50:420456+0000 2026-02-13T02:19:51.286 INFO:journalctl@ceph.mon.a.trial027.stdout: enable_multiple, ever_enabled_multiple: 1,1 2026-02-13T02:19:51.286 INFO:journalctl@ceph.mon.a.trial027.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2026-02-13T02:19:51.286 INFO:journalctl@ceph.mon.a.trial027.stdout: legacy client fscid: -1 2026-02-13T02:19:51.286 INFO:journalctl@ceph.mon.a.trial027.stdout: 2026-02-13T02:19:51.286 INFO:journalctl@ceph.mon.a.trial027.stdout: No filesystems configured 2026-02-13T02:19:51.286 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2026-02-13T02:19:51.286 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2026-02-13T02:19:51.286 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2026-02-13T02:19:51.287 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2026-02-13T02:19:51.287 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2026-02-13T02:19:51.287 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: mon.a is new leader, mons a in quorum (ranks 0) 2026-02-13T02:19:51.287 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: monmap epoch 1 2026-02-13T02:19:51.287 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: fsid 707dbb29-0882-11f1-9262-d404e6e7d460 2026-02-13T02:19:51.287 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: last_changed 2026-02-13T02:19:49.454661+0000 2026-02-13T02:19:51.287 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: created 2026-02-13T02:19:49.454661+0000 2026-02-13T02:19:51.287 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: min_mon_release 20 (tentacle) 2026-02-13T02:19:51.287 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: election_strategy: 1 2026-02-13T02:19:51.287 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: 0: [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] mon.a 2026-02-13T02:19:51.287 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: fsmap 2026-02-13T02:19:51.287 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: osdmap e1: 0 total, 0 up, 0 in 2026-02-13T02:19:51.287 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:51 trial027 ceph-mon[24567]: mgrmap e1: no daemons active 2026-02-13T02:19:51.328 INFO:teuthology.orchestra.run.trial027.stdout:Wrote config to /etc/ceph/ceph.conf 2026-02-13T02:19:51.329 INFO:teuthology.orchestra.run.trial027.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2026-02-13T02:19:51.329 INFO:teuthology.orchestra.run.trial027.stdout:Creating mgr... 2026-02-13T02:19:51.329 INFO:teuthology.orchestra.run.trial027.stdout:Verifying port 0.0.0.0:9283 ... 2026-02-13T02:19:51.330 INFO:teuthology.orchestra.run.trial027.stdout:Verifying port 0.0.0.0:8765 ... 2026-02-13T02:19:51.453 INFO:teuthology.orchestra.run.trial027.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mgr.x 2026-02-13T02:19:51.453 INFO:teuthology.orchestra.run.trial027.stdout:systemctl: stderr Failed to reset failed state of unit ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mgr.x.service: Unit ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mgr.x.service not loaded. 2026-02-13T02:19:51.564 INFO:teuthology.orchestra.run.trial027.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-707dbb29-0882-11f1-9262-d404e6e7d460.target.wants/ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mgr.x.service → /etc/systemd/system/ceph-707dbb29-0882-11f1-9262-d404e6e7d460@.service. 2026-02-13T02:19:51.703 INFO:teuthology.orchestra.run.trial027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2026-02-13T02:19:51.703 INFO:teuthology.orchestra.run.trial027.stdout:systemctl: stdout disabled 2026-02-13T02:19:51.709 INFO:teuthology.orchestra.run.trial027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2026-02-13T02:19:51.709 INFO:teuthology.orchestra.run.trial027.stdout:systemctl: stdout inactive 2026-02-13T02:19:51.709 INFO:teuthology.orchestra.run.trial027.stdout:firewalld.service is not enabled 2026-02-13T02:19:51.709 INFO:teuthology.orchestra.run.trial027.stdout:Not possible to enable service . firewalld.service is not available 2026-02-13T02:19:51.716 INFO:teuthology.orchestra.run.trial027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2026-02-13T02:19:51.716 INFO:teuthology.orchestra.run.trial027.stdout:systemctl: stdout disabled 2026-02-13T02:19:51.721 INFO:teuthology.orchestra.run.trial027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2026-02-13T02:19:51.722 INFO:teuthology.orchestra.run.trial027.stdout:systemctl: stdout inactive 2026-02-13T02:19:51.722 INFO:teuthology.orchestra.run.trial027.stdout:firewalld.service is not enabled 2026-02-13T02:19:51.722 INFO:teuthology.orchestra.run.trial027.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2026-02-13T02:19:51.722 INFO:teuthology.orchestra.run.trial027.stdout:Waiting for mgr to start... 2026-02-13T02:19:51.722 INFO:teuthology.orchestra.run.trial027.stdout:Waiting for mgr... 2026-02-13T02:19:51.900 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout 2026-02-13T02:19:51.900 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout { 2026-02-13T02:19:51.900 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "fsid": "707dbb29-0882-11f1-9262-d404e6e7d460", 2026-02-13T02:19:51.901 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "health": { 2026-02-13T02:19:51.901 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2026-02-13T02:19:51.901 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "checks": {}, 2026-02-13T02:19:51.901 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "mutes": [] 2026-02-13T02:19:51.901 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout }, 2026-02-13T02:19:51.901 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2026-02-13T02:19:51.901 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "quorum": [ 2026-02-13T02:19:51.901 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout 0 2026-02-13T02:19:51.901 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout ], 2026-02-13T02:19:51.901 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2026-02-13T02:19:51.901 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "a" 2026-02-13T02:19:51.901 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout ], 2026-02-13T02:19:51.901 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "quorum_age": 0, 2026-02-13T02:19:51.902 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "monmap": { 2026-02-13T02:19:51.902 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-13T02:19:51.902 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "tentacle", 2026-02-13T02:19:51.902 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2026-02-13T02:19:51.902 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout }, 2026-02-13T02:19:51.902 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "osdmap": { 2026-02-13T02:19:51.902 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-13T02:19:51.902 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2026-02-13T02:19:51.902 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2026-02-13T02:19:51.902 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2026-02-13T02:19:51.902 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2026-02-13T02:19:51.902 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2026-02-13T02:19:51.903 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2026-02-13T02:19:51.903 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout }, 2026-02-13T02:19:51.903 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "pgmap": { 2026-02-13T02:19:51.903 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2026-02-13T02:19:51.903 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2026-02-13T02:19:51.903 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2026-02-13T02:19:51.903 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2026-02-13T02:19:51.903 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2026-02-13T02:19:51.903 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2026-02-13T02:19:51.903 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2026-02-13T02:19:51.903 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2026-02-13T02:19:51.903 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout }, 2026-02-13T02:19:51.903 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "fsmap": { 2026-02-13T02:19:51.904 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-13T02:19:51.904 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "btime": "2026-02-13T02:19:50:420456+0000", 2026-02-13T02:19:51.904 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2026-02-13T02:19:51.904 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2026-02-13T02:19:51.904 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout }, 2026-02-13T02:19:51.904 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2026-02-13T02:19:51.904 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "available": false, 2026-02-13T02:19:51.904 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2026-02-13T02:19:51.904 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "modules": [ 2026-02-13T02:19:51.904 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "iostat", 2026-02-13T02:19:51.904 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "nfs" 2026-02-13T02:19:51.904 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout ], 2026-02-13T02:19:51.904 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "services": {} 2026-02-13T02:19:51.905 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout }, 2026-02-13T02:19:51.905 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "servicemap": { 2026-02-13T02:19:51.905 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-13T02:19:51.905 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "modified": "2026-02-13T02:19:50.420783+0000", 2026-02-13T02:19:51.905 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "services": {} 2026-02-13T02:19:51.905 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout }, 2026-02-13T02:19:51.905 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2026-02-13T02:19:51.905 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout } 2026-02-13T02:19:51.905 INFO:teuthology.orchestra.run.trial027.stdout:mgr not available, waiting (1/15)... 2026-02-13T02:19:52.089 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:52 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:52.087+0000 7f853006cfc0 -1 mgr[py] Loading module diskprediction_local in main interpreter 2026-02-13T02:19:52.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:52 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1833793655' entity='client.admin' 2026-02-13T02:19:52.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:52 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2955437277' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2026-02-13T02:19:52.409 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:52 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:52.124+0000 7f853006cfc0 -1 mgr[py] Loading module status in main interpreter 2026-02-13T02:19:52.409 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:52 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:52.126+0000 7f853006cfc0 -1 mgr[py] Loading module rgw in main interpreter 2026-02-13T02:19:52.409 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:52 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:52.130+0000 7f853006cfc0 -1 mgr[py] Loading module insights in main interpreter 2026-02-13T02:19:52.409 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:52 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:52.132+0000 7f853006cfc0 -1 mgr[py] Loading module iostat in main interpreter 2026-02-13T02:19:52.409 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:52 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:52.134+0000 7f853006cfc0 -1 mgr[py] Loading module pg_autoscaler in main interpreter 2026-02-13T02:19:52.409 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:52 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:52.137+0000 7f853006cfc0 -1 mgr[py] Loading module osd_perf_query in main interpreter 2026-02-13T02:19:52.409 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:52 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:52.137+0000 7f853006cfc0 -1 mgr[py] Loading module crash in main interpreter 2026-02-13T02:19:52.409 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:52 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:52.141+0000 7f853006cfc0 -1 mgr[py] Loading module progress in main interpreter 2026-02-13T02:19:52.409 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:52 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:52.143+0000 7f853006cfc0 -1 mgr[py] Loading module rook in main interpreter 2026-02-13T02:19:52.409 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:52 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:52.315+0000 7f853006cfc0 -1 mgr[py] Loading module nfs in main interpreter 2026-02-13T02:19:52.410 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:52 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:52.315+0000 7f853006cfc0 -1 mgr[py] Loading module telegraf in main interpreter 2026-02-13T02:19:52.410 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:52 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:52.317+0000 7f853006cfc0 -1 mgr[py] Loading module selftest in main interpreter 2026-02-13T02:19:52.410 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:52 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:52.319+0000 7f853006cfc0 -1 mgr[py] Loading module test_orchestrator in main interpreter 2026-02-13T02:19:52.410 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:52 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:52.321+0000 7f853006cfc0 -1 mgr[py] Loading module osd_support in main interpreter 2026-02-13T02:19:52.410 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:52 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:52.321+0000 7f853006cfc0 -1 mgr[py] Loading module orchestrator in main interpreter 2026-02-13T02:19:52.410 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:52 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:52.321+0000 7f853006cfc0 -1 mgr[py] Loading module cephadm in main interpreter 2026-02-13T02:19:52.908 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:52 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:52.465+0000 7f853006cfc0 -1 mgr[py] Loading module volumes in main interpreter 2026-02-13T02:19:52.908 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:52 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:52.493+0000 7f853006cfc0 -1 mgr[py] Loading module k8sevents in main interpreter 2026-02-13T02:19:52.908 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:52 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:52.497+0000 7f853006cfc0 -1 mgr[py] Loading module mds_autoscaler in main interpreter 2026-02-13T02:19:52.909 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:52 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:52.498+0000 7f853006cfc0 -1 mgr[py] Loading module localpool in main interpreter 2026-02-13T02:19:52.909 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:52 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:52.499+0000 7f853006cfc0 -1 mgr[py] Loading module influx in main interpreter 2026-02-13T02:19:53.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:53 trial027 ceph-mon[24567]: Activating manager daemon x 2026-02-13T02:19:53.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:53 trial027 ceph-mon[24567]: mgrmap e2: x(active, starting, since 0.002547s) 2026-02-13T02:19:53.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:53 trial027 ceph-mon[24567]: from='mgr.14100 10.20.193.27:0/371423872' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2026-02-13T02:19:53.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:53 trial027 ceph-mon[24567]: from='mgr.14100 10.20.193.27:0/371423872' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2026-02-13T02:19:53.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:53 trial027 ceph-mon[24567]: from='mgr.14100 10.20.193.27:0/371423872' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2026-02-13T02:19:53.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:53 trial027 ceph-mon[24567]: from='mgr.14100 10.20.193.27:0/371423872' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-13T02:19:53.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:53 trial027 ceph-mon[24567]: from='mgr.14100 10.20.193.27:0/371423872' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2026-02-13T02:19:53.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:53 trial027 ceph-mon[24567]: Manager daemon x is now available 2026-02-13T02:19:53.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:53 trial027 ceph-mon[24567]: from='mgr.14100 10.20.193.27:0/371423872' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2026-02-13T02:19:53.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:53 trial027 ceph-mon[24567]: from='mgr.14100 10.20.193.27:0/371423872' entity='mgr.x' 2026-02-13T02:19:53.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:53 trial027 ceph-mon[24567]: from='mgr.14100 10.20.193.27:0/371423872' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2026-02-13T02:19:53.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:53 trial027 ceph-mon[24567]: from='mgr.14100 10.20.193.27:0/371423872' entity='mgr.x' 2026-02-13T02:19:53.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:53 trial027 ceph-mon[24567]: from='mgr.14100 10.20.193.27:0/371423872' entity='mgr.x' 2026-02-13T02:19:54.172 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout 2026-02-13T02:19:54.172 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout { 2026-02-13T02:19:54.172 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "fsid": "707dbb29-0882-11f1-9262-d404e6e7d460", 2026-02-13T02:19:54.172 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "health": { 2026-02-13T02:19:54.172 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2026-02-13T02:19:54.172 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "checks": {}, 2026-02-13T02:19:54.172 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "mutes": [] 2026-02-13T02:19:54.172 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout }, 2026-02-13T02:19:54.173 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2026-02-13T02:19:54.173 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "quorum": [ 2026-02-13T02:19:54.173 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout 0 2026-02-13T02:19:54.173 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout ], 2026-02-13T02:19:54.173 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2026-02-13T02:19:54.173 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "a" 2026-02-13T02:19:54.173 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout ], 2026-02-13T02:19:54.173 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2026-02-13T02:19:54.173 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "monmap": { 2026-02-13T02:19:54.173 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-13T02:19:54.173 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "tentacle", 2026-02-13T02:19:54.173 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2026-02-13T02:19:54.174 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout }, 2026-02-13T02:19:54.174 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "osdmap": { 2026-02-13T02:19:54.174 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-13T02:19:54.174 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2026-02-13T02:19:54.174 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2026-02-13T02:19:54.174 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2026-02-13T02:19:54.174 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2026-02-13T02:19:54.174 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2026-02-13T02:19:54.174 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2026-02-13T02:19:54.174 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout }, 2026-02-13T02:19:54.174 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "pgmap": { 2026-02-13T02:19:54.174 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2026-02-13T02:19:54.175 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2026-02-13T02:19:54.175 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2026-02-13T02:19:54.175 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2026-02-13T02:19:54.175 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2026-02-13T02:19:54.175 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2026-02-13T02:19:54.175 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2026-02-13T02:19:54.175 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2026-02-13T02:19:54.175 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout }, 2026-02-13T02:19:54.175 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "fsmap": { 2026-02-13T02:19:54.175 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-13T02:19:54.175 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "btime": "2026-02-13T02:19:50:420456+0000", 2026-02-13T02:19:54.175 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2026-02-13T02:19:54.175 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2026-02-13T02:19:54.176 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout }, 2026-02-13T02:19:54.176 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2026-02-13T02:19:54.176 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "available": true, 2026-02-13T02:19:54.176 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2026-02-13T02:19:54.176 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "modules": [ 2026-02-13T02:19:54.176 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "iostat", 2026-02-13T02:19:54.176 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "nfs" 2026-02-13T02:19:54.176 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout ], 2026-02-13T02:19:54.176 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "services": {} 2026-02-13T02:19:54.176 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout }, 2026-02-13T02:19:54.176 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "servicemap": { 2026-02-13T02:19:54.176 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-13T02:19:54.176 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "modified": "2026-02-13T02:19:50.420783+0000", 2026-02-13T02:19:54.177 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "services": {} 2026-02-13T02:19:54.177 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout }, 2026-02-13T02:19:54.177 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2026-02-13T02:19:54.177 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout } 2026-02-13T02:19:54.177 INFO:teuthology.orchestra.run.trial027.stdout:mgr is available 2026-02-13T02:19:54.395 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout 2026-02-13T02:19:54.395 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout [global] 2026-02-13T02:19:54.395 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout fsid = 707dbb29-0882-11f1-9262-d404e6e7d460 2026-02-13T02:19:54.395 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2026-02-13T02:19:54.395 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout mon_host = [v2:10.20.193.27:3300,v1:10.20.193.27:6789] 2026-02-13T02:19:54.395 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2026-02-13T02:19:54.395 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2026-02-13T02:19:54.395 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2026-02-13T02:19:54.396 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2026-02-13T02:19:54.396 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout 2026-02-13T02:19:54.396 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout [mgr] 2026-02-13T02:19:54.396 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2026-02-13T02:19:54.396 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout 2026-02-13T02:19:54.396 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout [osd] 2026-02-13T02:19:54.396 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2026-02-13T02:19:54.396 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2026-02-13T02:19:54.396 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2026-02-13T02:19:54.396 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2026-02-13T02:19:54.396 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2026-02-13T02:19:54.396 INFO:teuthology.orchestra.run.trial027.stdout:Enabling cephadm module... 2026-02-13T02:19:54.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:54 trial027 ceph-mon[24567]: mgrmap e3: x(active, since 1.00516s) 2026-02-13T02:19:54.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:54 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1632962496' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2026-02-13T02:19:54.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:54 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1089284944' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2026-02-13T02:19:55.776 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:55 trial027 ceph-mon[24567]: mgrmap e4: x(active, since 2s) 2026-02-13T02:19:55.776 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:55 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1347469330' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2026-02-13T02:19:55.777 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:55 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: ignoring --setuser ceph since I am not root 2026-02-13T02:19:55.777 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:55 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: ignoring --setgroup ceph since I am not root 2026-02-13T02:19:55.777 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:55 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:55.545+0000 7f953c86afc0 -1 mgr[py] Loading module stats in main interpreter 2026-02-13T02:19:55.777 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:55 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:55.584+0000 7f953c86afc0 -1 mgr[py] Loading module dashboard in main interpreter 2026-02-13T02:19:55.802 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout { 2026-02-13T02:19:55.802 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "epoch": 5, 2026-02-13T02:19:55.802 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "available": true, 2026-02-13T02:19:55.802 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "active_name": "x", 2026-02-13T02:19:55.803 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "num_standby": 0 2026-02-13T02:19:55.803 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout } 2026-02-13T02:19:55.803 INFO:teuthology.orchestra.run.trial027.stdout:Waiting for the mgr to restart... 2026-02-13T02:19:55.803 INFO:teuthology.orchestra.run.trial027.stdout:Waiting for mgr epoch 5... 2026-02-13T02:19:56.125 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:55 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:55.837+0000 7f953c86afc0 -1 mgr[py] Loading module mirroring in main interpreter 2026-02-13T02:19:56.125 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:55 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:55.846+0000 7f953c86afc0 -1 mgr[py] Loading module snap_schedule in main interpreter 2026-02-13T02:19:56.125 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:55 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:55.853+0000 7f953c86afc0 -1 mgr[py] Loading module prometheus in main interpreter 2026-02-13T02:19:56.126 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:55 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:55.859+0000 7f953c86afc0 -1 mgr[py] Loading module balancer in main interpreter 2026-02-13T02:19:56.126 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:55 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:55.865+0000 7f953c86afc0 -1 mgr[py] Loading module rbd_support in main interpreter 2026-02-13T02:19:56.126 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:55 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:55.877+0000 7f953c86afc0 -1 mgr[py] Loading module smb in main interpreter 2026-02-13T02:19:56.126 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:55 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:55.902+0000 7f953c86afc0 -1 mgr[py] Loading module devicehealth in main interpreter 2026-02-13T02:19:56.126 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:55 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:55.905+0000 7f953c86afc0 -1 mgr[py] Loading module alerts in main interpreter 2026-02-13T02:19:56.126 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:55 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:55.907+0000 7f953c86afc0 -1 mgr[py] Loading module telemetry in main interpreter 2026-02-13T02:19:56.126 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:55 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:55.913+0000 7f953c86afc0 -1 mgr[py] Loading module diskprediction_local in main interpreter 2026-02-13T02:19:56.126 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:55 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:55.949+0000 7f953c86afc0 -1 mgr[py] Loading module status in main interpreter 2026-02-13T02:19:56.126 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:55 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:55.951+0000 7f953c86afc0 -1 mgr[py] Loading module rgw in main interpreter 2026-02-13T02:19:56.126 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:55 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:55.955+0000 7f953c86afc0 -1 mgr[py] Loading module insights in main interpreter 2026-02-13T02:19:56.126 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:55 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:55.957+0000 7f953c86afc0 -1 mgr[py] Loading module iostat in main interpreter 2026-02-13T02:19:56.126 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:55 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:55.958+0000 7f953c86afc0 -1 mgr[py] Loading module pg_autoscaler in main interpreter 2026-02-13T02:19:56.127 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:55 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:55.961+0000 7f953c86afc0 -1 mgr[py] Loading module osd_perf_query in main interpreter 2026-02-13T02:19:56.127 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:55 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:55.961+0000 7f953c86afc0 -1 mgr[py] Loading module crash in main interpreter 2026-02-13T02:19:56.127 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:55 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:55.964+0000 7f953c86afc0 -1 mgr[py] Loading module progress in main interpreter 2026-02-13T02:19:56.127 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:55 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:55.967+0000 7f953c86afc0 -1 mgr[py] Loading module rook in main interpreter 2026-02-13T02:19:56.127 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:56 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:56.124+0000 7f953c86afc0 -1 mgr[py] Loading module nfs in main interpreter 2026-02-13T02:19:56.127 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:56 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:56.124+0000 7f953c86afc0 -1 mgr[py] Loading module telegraf in main interpreter 2026-02-13T02:19:56.408 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:56 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:56.126+0000 7f953c86afc0 -1 mgr[py] Loading module selftest in main interpreter 2026-02-13T02:19:56.408 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:56 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:56.129+0000 7f953c86afc0 -1 mgr[py] Loading module test_orchestrator in main interpreter 2026-02-13T02:19:56.408 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:56 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:56.130+0000 7f953c86afc0 -1 mgr[py] Loading module osd_support in main interpreter 2026-02-13T02:19:56.408 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:56 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:56.131+0000 7f953c86afc0 -1 mgr[py] Loading module orchestrator in main interpreter 2026-02-13T02:19:56.409 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:56 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:56.131+0000 7f953c86afc0 -1 mgr[py] Loading module cephadm in main interpreter 2026-02-13T02:19:56.409 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:56 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:56.272+0000 7f953c86afc0 -1 mgr[py] Loading module volumes in main interpreter 2026-02-13T02:19:56.409 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:56 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:56.300+0000 7f953c86afc0 -1 mgr[py] Loading module k8sevents in main interpreter 2026-02-13T02:19:56.409 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:56 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:56.304+0000 7f953c86afc0 -1 mgr[py] Loading module mds_autoscaler in main interpreter 2026-02-13T02:19:56.409 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:56 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:56.305+0000 7f953c86afc0 -1 mgr[py] Loading module localpool in main interpreter 2026-02-13T02:19:56.409 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:56 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:19:56.306+0000 7f953c86afc0 -1 mgr[py] Loading module influx in main interpreter 2026-02-13T02:19:56.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:56 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1347469330' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2026-02-13T02:19:56.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:56 trial027 ceph-mon[24567]: mgrmap e5: x(active, since 3s) 2026-02-13T02:19:56.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:56 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/152052800' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2026-02-13T02:19:56.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:56 trial027 ceph-mon[24567]: Active manager daemon x restarted 2026-02-13T02:19:56.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:56 trial027 ceph-mon[24567]: Activating manager daemon x 2026-02-13T02:19:56.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:56 trial027 ceph-mon[24567]: osdmap e2: 0 total, 0 up, 0 in 2026-02-13T02:19:56.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:56 trial027 ceph-mon[24567]: mgrmap e6: x(active, starting, since 0.00278081s) 2026-02-13T02:19:56.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:56 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-13T02:19:56.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:56 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2026-02-13T02:19:56.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:56 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2026-02-13T02:19:56.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:56 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2026-02-13T02:19:56.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:56 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2026-02-13T02:19:56.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:56 trial027 ceph-mon[24567]: Manager daemon x is now available 2026-02-13T02:19:56.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:56 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:19:56.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:56 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:19:56.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:56 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:19:56.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:56 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:19:56.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:56 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:19:56.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:56 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:19:56.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:56 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:19:57.340 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout { 2026-02-13T02:19:57.340 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2026-02-13T02:19:57.341 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout "initialized": true 2026-02-13T02:19:57.341 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout } 2026-02-13T02:19:57.341 INFO:teuthology.orchestra.run.trial027.stdout:mgr epoch 5 is available 2026-02-13T02:19:57.341 INFO:teuthology.orchestra.run.trial027.stdout:Verifying orchestrator module is enabled... 2026-02-13T02:19:58.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:19:58.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:19:58.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:19:58.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2026-02-13T02:19:58.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2026-02-13T02:19:58.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:19:58.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:57 trial027 ceph-mon[24567]: mgrmap e7: x(active, since 1.00487s) 2026-02-13T02:19:58.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:57 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/243079264' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch 2026-02-13T02:19:58.340 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stderr module 'orchestrator' is already enabled (always-on) 2026-02-13T02:19:58.340 INFO:teuthology.orchestra.run.trial027.stdout:Setting orchestrator backend to cephadm... 2026-02-13T02:19:58.794 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout value unchanged 2026-02-13T02:19:58.795 INFO:teuthology.orchestra.run.trial027.stdout:Generating ssh key... 2026-02-13T02:19:59.143 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:59 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: Generating public/private ed25519 key pair. 2026-02-13T02:19:59.143 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:59 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: Your identification has been saved in /tmp/tmp3b83543j/key 2026-02-13T02:19:59.143 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:59 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: Your public key has been saved in /tmp/tmp3b83543j/key.pub 2026-02-13T02:19:59.143 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:59 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: The key fingerprint is: 2026-02-13T02:19:59.143 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:59 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: SHA256:UKcace7F+oixwhxnRORBK3AIiOI80cQUFgdmCIOviNE ceph-707dbb29-0882-11f1-9262-d404e6e7d460 2026-02-13T02:19:59.143 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:59 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: The key's randomart image is: 2026-02-13T02:19:59.143 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:59 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: +--[ED25519 256]--+ 2026-02-13T02:19:59.143 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:59 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: |O.B@=+* o . | 2026-02-13T02:19:59.143 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:59 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: |=+==.o B + | 2026-02-13T02:19:59.143 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:59 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: |+o. . * o o | 2026-02-13T02:19:59.144 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:59 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: |.+E o = o | 2026-02-13T02:19:59.144 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:59 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: |oo. . = S | 2026-02-13T02:19:59.144 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:59 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: |+ o + + o | 2026-02-13T02:19:59.144 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:59 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: | + o . . | 2026-02-13T02:19:59.144 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:59 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: | . | 2026-02-13T02:19:59.144 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:59 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: | | 2026-02-13T02:19:59.144 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:19:59 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: +----[SHA256]-----+ 2026-02-13T02:19:59.255 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIF8hk9njdxeq+vFUwz30mRkPfpTUAX0blc+jmrTq8JJi ceph-707dbb29-0882-11f1-9262-d404e6e7d460 2026-02-13T02:19:59.256 INFO:teuthology.orchestra.run.trial027.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2026-02-13T02:19:59.256 INFO:teuthology.orchestra.run.trial027.stdout:Adding key to root@localhost authorized_keys... 2026-02-13T02:19:59.256 INFO:teuthology.orchestra.run.trial027.stdout:Adding host trial027... 2026-02-13T02:19:59.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:59 trial027 ceph-mon[24567]: Optimize plan auto_2026-02-13_02:19:56, mode upmap, max misplaced 0.050000 2026-02-13T02:19:59.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:59 trial027 ceph-mon[24567]: No pools available 2026-02-13T02:19:59.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:59 trial027 ceph-mon[24567]: Found migration_current of "None". Setting to last migration. 2026-02-13T02:19:59.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:59 trial027 ceph-mon[24567]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2026-02-13T02:19:59.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:59 trial027 ceph-mon[24567]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2026-02-13T02:19:59.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:59 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/243079264' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished 2026-02-13T02:19:59.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:59 trial027 ceph-mon[24567]: mgrmap e8: x(active, since 2s) 2026-02-13T02:19:59.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:59 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:19:59.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:59 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:19:59.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:59 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:19:59.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:19:59 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:00.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:00 trial027 ceph-mon[24567]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:20:00.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:00 trial027 ceph-mon[24567]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:20:00.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:00 trial027 ceph-mon[24567]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:20:00.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:00 trial027 ceph-mon[24567]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:20:00.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:00 trial027 ceph-mon[24567]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "trial027", "addr": "10.20.193.27", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:20:01.319 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout Added host 'trial027' with addr '10.20.193.27' 2026-02-13T02:20:01.320 INFO:teuthology.orchestra.run.trial027.stdout:Deploying unmanaged mon service... 2026-02-13T02:20:01.563 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2026-02-13T02:20:01.563 INFO:teuthology.orchestra.run.trial027.stdout:Deploying unmanaged mgr service... 2026-02-13T02:20:01.816 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2026-02-13T02:20:02.046 INFO:teuthology.orchestra.run.trial027.stdout:Saving cluster configuration to /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/config directory 2026-02-13T02:20:02.328 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:02 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:02.329 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:02 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:02.329 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:02 trial027 ceph-mon[24567]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:20:02.329 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:02 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:02.329 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:02 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:02.329 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:02 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:02.329 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:02 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2881815741' entity='client.admin' 2026-02-13T02:20:02.329 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:02 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:02.341 INFO:teuthology.orchestra.run.trial027.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2026-02-13T02:20:02.341 INFO:teuthology.orchestra.run.trial027.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2026-02-13T02:20:02.341 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:20:02.342 INFO:teuthology.orchestra.run.trial027.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2026-02-13T02:20:02.342 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:20:02.342 INFO:teuthology.orchestra.run.trial027.stdout:Or, if you are only running a single cluster on this host: 2026-02-13T02:20:02.342 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:20:02.342 INFO:teuthology.orchestra.run.trial027.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2026-02-13T02:20:02.342 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:20:02.342 INFO:teuthology.orchestra.run.trial027.stdout:Please consider enabling telemetry to help improve Ceph: 2026-02-13T02:20:02.342 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:20:02.342 INFO:teuthology.orchestra.run.trial027.stdout: ceph telemetry on 2026-02-13T02:20:02.342 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:20:02.342 INFO:teuthology.orchestra.run.trial027.stdout:For more information see: 2026-02-13T02:20:02.343 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:20:02.343 INFO:teuthology.orchestra.run.trial027.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2026-02-13T02:20:02.343 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:20:02.343 INFO:teuthology.orchestra.run.trial027.stdout:Bootstrap complete. 2026-02-13T02:20:02.349 INFO:teuthology.orchestra.run.trial027.stdout:systemctl: stdout static 2026-02-13T02:20:02.355 INFO:teuthology.orchestra.run.trial027.stdout:Non-zero exit code 3 from systemctl is-active logrotate 2026-02-13T02:20:02.355 INFO:teuthology.orchestra.run.trial027.stdout:systemctl: stdout inactive 2026-02-13T02:20:02.355 INFO:teuthology.orchestra.run.trial027.stdout:Enabling the logrotate.timer service to perform daily log rotation. 2026-02-13T02:20:02.504 INFO:tasks.cephadm:Fetching config... 2026-02-13T02:20:02.504 DEBUG:teuthology.orchestra.run.trial027:> set -ex 2026-02-13T02:20:02.504 DEBUG:teuthology.orchestra.run.trial027:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2026-02-13T02:20:02.521 INFO:tasks.cephadm:Fetching client.admin keyring... 2026-02-13T02:20:02.522 DEBUG:teuthology.orchestra.run.trial027:> set -ex 2026-02-13T02:20:02.522 DEBUG:teuthology.orchestra.run.trial027:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2026-02-13T02:20:02.578 INFO:tasks.cephadm:Fetching mon keyring... 2026-02-13T02:20:02.578 DEBUG:teuthology.orchestra.run.trial027:> set -ex 2026-02-13T02:20:02.578 DEBUG:teuthology.orchestra.run.trial027:> sudo dd if=/var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/keyring of=/dev/stdout 2026-02-13T02:20:02.644 INFO:tasks.cephadm:Fetching pub ssh key... 2026-02-13T02:20:02.644 DEBUG:teuthology.orchestra.run.trial027:> set -ex 2026-02-13T02:20:02.644 DEBUG:teuthology.orchestra.run.trial027:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2026-02-13T02:20:02.702 INFO:tasks.cephadm:Installing pub ssh key for root users... 2026-02-13T02:20:02.702 DEBUG:teuthology.orchestra.run.trial027:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIF8hk9njdxeq+vFUwz30mRkPfpTUAX0blc+jmrTq8JJi ceph-707dbb29-0882-11f1-9262-d404e6e7d460' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2026-02-13T02:20:02.775 INFO:teuthology.orchestra.run.trial027.stdout:ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIF8hk9njdxeq+vFUwz30mRkPfpTUAX0blc+jmrTq8JJi ceph-707dbb29-0882-11f1-9262-d404e6e7d460 2026-02-13T02:20:02.785 DEBUG:teuthology.orchestra.run.trial028:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIF8hk9njdxeq+vFUwz30mRkPfpTUAX0blc+jmrTq8JJi ceph-707dbb29-0882-11f1-9262-d404e6e7d460' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2026-02-13T02:20:02.820 INFO:teuthology.orchestra.run.trial028.stdout:ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIF8hk9njdxeq+vFUwz30mRkPfpTUAX0blc+jmrTq8JJi ceph-707dbb29-0882-11f1-9262-d404e6e7d460 2026-02-13T02:20:02.829 DEBUG:teuthology.orchestra.run.trial107:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIF8hk9njdxeq+vFUwz30mRkPfpTUAX0blc+jmrTq8JJi ceph-707dbb29-0882-11f1-9262-d404e6e7d460' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2026-02-13T02:20:02.864 INFO:teuthology.orchestra.run.trial107.stdout:ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIF8hk9njdxeq+vFUwz30mRkPfpTUAX0blc+jmrTq8JJi ceph-707dbb29-0882-11f1-9262-d404e6e7d460 2026-02-13T02:20:02.875 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2026-02-13T02:20:02.990 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:03.273 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2026-02-13T02:20:03.273 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph orch client-keyring set client.admin '*' --mode 0755 2026-02-13T02:20:03.391 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:03.506 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:03 trial027 ceph-mon[24567]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:20:03.507 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:03 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/750692452' entity='client.admin' 2026-02-13T02:20:03.507 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:03 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:03.507 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:03 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3101809786' entity='client.admin' 2026-02-13T02:20:03.653 INFO:tasks.cephadm:Writing (initial) conf and keyring to trial028 2026-02-13T02:20:03.653 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-13T02:20:03.653 DEBUG:teuthology.orchestra.run.trial028:> dd of=/etc/ceph/ceph.conf 2026-02-13T02:20:03.669 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-13T02:20:03.669 DEBUG:teuthology.orchestra.run.trial028:> dd of=/etc/ceph/ceph.client.admin.keyring 2026-02-13T02:20:03.726 INFO:tasks.cephadm:Adding host trial028 to orchestrator... 2026-02-13T02:20:03.727 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph orch host add trial028 2026-02-13T02:20:03.852 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:04.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:04 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:04.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:04 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:04.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:04 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:04.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:04 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:04.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:04 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:04.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:04 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:04.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:04 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:trial027", "name": "osd_memory_target"} : dispatch 2026-02-13T02:20:04.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:04 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:04.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:04 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:05.891 INFO:teuthology.orchestra.run.trial027.stdout:Added host 'trial028' with addr '10.20.193.28' 2026-02-13T02:20:05.892 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:05 trial027 ceph-mon[24567]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:20:05.892 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:05 trial027 ceph-mon[24567]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "trial028", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:20:05.892 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:05 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:05.892 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:05 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:05.892 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:05 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:05.892 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:05 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:05.892 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:05 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:05.892 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:05 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:05.893 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:05 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:05.938 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph orch host ls --format=json 2026-02-13T02:20:06.057 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:06.273 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:20:06.273 INFO:teuthology.orchestra.run.trial027.stdout:[{"addr": "10.20.193.27", "hostname": "trial027", "labels": [], "status": ""}, {"addr": "10.20.193.28", "hostname": "trial028", "labels": [], "status": ""}] 2026-02-13T02:20:06.314 INFO:tasks.cephadm:Writing (initial) conf and keyring to trial107 2026-02-13T02:20:06.315 DEBUG:teuthology.orchestra.run.trial107:> set -ex 2026-02-13T02:20:06.315 DEBUG:teuthology.orchestra.run.trial107:> dd of=/etc/ceph/ceph.conf 2026-02-13T02:20:06.333 DEBUG:teuthology.orchestra.run.trial107:> set -ex 2026-02-13T02:20:06.333 DEBUG:teuthology.orchestra.run.trial107:> dd of=/etc/ceph/ceph.client.admin.keyring 2026-02-13T02:20:06.391 INFO:tasks.cephadm:Adding host trial107 to orchestrator... 2026-02-13T02:20:06.391 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph orch host add trial107 2026-02-13T02:20:06.513 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:07.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:06 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:07.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:06 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:07.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:06 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:07.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:06 trial027 ceph-mon[24567]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-13T02:20:07.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:06 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:08.543 INFO:teuthology.orchestra.run.trial027.stdout:Added host 'trial107' with addr '10.20.193.107' 2026-02-13T02:20:08.543 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:08 trial027 ceph-mon[24567]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "trial107", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:20:08.543 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:08 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:08.543 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:08 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:08.543 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:08 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:08.543 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:08 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:08.544 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:08 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:08.544 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:08 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:trial028", "name": "osd_memory_target"} : dispatch 2026-02-13T02:20:08.544 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:08 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:08.544 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:08 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:08.589 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph orch host ls --format=json 2026-02-13T02:20:08.706 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:08.918 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:20:08.918 INFO:teuthology.orchestra.run.trial027.stdout:[{"addr": "10.20.193.27", "hostname": "trial027", "labels": [], "status": ""}, {"addr": "10.20.193.28", "hostname": "trial028", "labels": [], "status": ""}, {"addr": "10.20.193.107", "hostname": "trial107", "labels": [], "status": ""}] 2026-02-13T02:20:08.980 INFO:tasks.cephadm:Setting crush tunables to default 2026-02-13T02:20:08.980 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd crush tunables default 2026-02-13T02:20:09.096 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:09.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:09 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:09.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:09 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:09.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:09 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:09.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:09 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:09.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:09 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:09.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:09 trial027 ceph-mon[24567]: from='client.14158 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-13T02:20:09.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:09 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:09.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:09 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:09.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:09 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2075626208' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2026-02-13T02:20:10.220 INFO:teuthology.orchestra.run.trial027.stderr:adjusted tunables profile to default 2026-02-13T02:20:10.266 INFO:tasks.cephadm:Adding mon.a on trial027 2026-02-13T02:20:10.266 INFO:tasks.cephadm:Adding mon.b on trial028 2026-02-13T02:20:10.267 INFO:tasks.cephadm:Adding mon.c on trial107 2026-02-13T02:20:10.267 DEBUG:teuthology.orchestra.run.trial107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph orch apply mon '3;trial027:10.20.193.27=a;trial028:10.20.193.28=b;trial107:10.20.193.107=c' 2026-02-13T02:20:10.399 INFO:teuthology.orchestra.run.trial107.stderr:Inferring config /etc/ceph/ceph.conf 2026-02-13T02:20:10.399 INFO:teuthology.orchestra.run.trial107.stderr:Inferring config /etc/ceph/ceph.conf 2026-02-13T02:20:10.620 INFO:teuthology.orchestra.run.trial107.stdout:Scheduled mon update... 2026-02-13T02:20:10.660 DEBUG:teuthology.orchestra.run.trial028:mon.b> sudo journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mon.b.service 2026-02-13T02:20:10.663 DEBUG:teuthology.orchestra.run.trial107:mon.c> sudo journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mon.c.service 2026-02-13T02:20:10.666 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2026-02-13T02:20:10.666 DEBUG:teuthology.orchestra.run.trial107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph mon dump -f json 2026-02-13T02:20:10.941 INFO:teuthology.orchestra.run.trial107.stderr:Inferring config /etc/ceph/ceph.conf 2026-02-13T02:20:10.941 INFO:teuthology.orchestra.run.trial107.stderr:Inferring config /etc/ceph/ceph.conf 2026-02-13T02:20:11.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:10 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:11.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:10 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2075626208' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2026-02-13T02:20:11.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:10 trial027 ceph-mon[24567]: osdmap e3: 0 total, 0 up, 0 in 2026-02-13T02:20:11.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:10 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:11.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:10 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:11.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:10 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:11.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:10 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:11.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:10 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:11.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:10 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:trial107", "name": "osd_memory_target"} : dispatch 2026-02-13T02:20:11.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:10 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:11.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:10 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:11.199 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:20:11.199 INFO:teuthology.orchestra.run.trial107.stdout:{"epoch":1,"fsid":"707dbb29-0882-11f1-9262-d404e6e7d460","modified":"2026-02-13T02:19:49.454661Z","created":"2026-02-13T02:19:49.454661Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:3300","nonce":0},{"type":"v1","addr":"10.20.193.27:6789","nonce":0}]},"addr":"10.20.193.27:6789/0","public_addr":"10.20.193.27:6789/0","priority":0,"weight":0,"time_added":"2026-02-13T02:19:49:454669+0000","crush_location":[]}],"quorum":[0]} 2026-02-13T02:20:11.200 INFO:teuthology.orchestra.run.trial107.stderr:dumped monmap epoch 1 2026-02-13T02:20:12.240 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2026-02-13T02:20:12.240 DEBUG:teuthology.orchestra.run.trial107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph mon dump -f json 2026-02-13T02:20:12.456 INFO:teuthology.orchestra.run.trial107.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.c/config 2026-02-13T02:20:12.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:12 trial027 ceph-mon[24567]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;trial027:10.20.193.27=a;trial028:10.20.193.28=b;trial107:10.20.193.107=c", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:20:12.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:12 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:12.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:12 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:12.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:12 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:12.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:12 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2026-02-13T02:20:12.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:12 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:12.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:12 trial027 ceph-mon[24567]: from='client.? 10.20.193.107:0/3744359963' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2026-02-13T02:20:17.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:17 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-13T02:20:17.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:17 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-13T02:20:17.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:17 trial027 ceph-mon[24567]: mon.a calling monitor election 2026-02-13T02:20:17.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:17 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-13T02:20:17.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:17 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-13T02:20:17.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:17 trial027 ceph-mon[24567]: mon.c calling monitor election 2026-02-13T02:20:17.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:17 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-13T02:20:17.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:17 trial027 ceph-mon[24567]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:17.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:17 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-13T02:20:17.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:17 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-13T02:20:17.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:17 trial027 ceph-mon[24567]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2026-02-13T02:20:17.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:17 trial027 ceph-mon[24567]: monmap epoch 2 2026-02-13T02:20:17.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:17 trial027 ceph-mon[24567]: fsid 707dbb29-0882-11f1-9262-d404e6e7d460 2026-02-13T02:20:17.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:17 trial027 ceph-mon[24567]: last_changed 2026-02-13T02:20:12.541886+0000 2026-02-13T02:20:17.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:17 trial027 ceph-mon[24567]: created 2026-02-13T02:19:49.454661+0000 2026-02-13T02:20:17.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:17 trial027 ceph-mon[24567]: min_mon_release 20 (tentacle) 2026-02-13T02:20:17.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:17 trial027 ceph-mon[24567]: election_strategy: 1 2026-02-13T02:20:17.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:17 trial027 ceph-mon[24567]: 0: [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] mon.a 2026-02-13T02:20:17.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:17 trial027 ceph-mon[24567]: 1: [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] mon.c 2026-02-13T02:20:17.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:17 trial027 ceph-mon[24567]: fsmap 2026-02-13T02:20:17.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:17 trial027 ceph-mon[24567]: osdmap e3: 0 total, 0 up, 0 in 2026-02-13T02:20:17.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:17 trial027 ceph-mon[24567]: mgrmap e8: x(active, since 21s) 2026-02-13T02:20:17.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:17 trial027 ceph-mon[24567]: overall HEALTH_OK 2026-02-13T02:20:17.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:17 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:17.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:17 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:17.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:17 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:17.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:17 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2026-02-13T02:20:17.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:17 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:18.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:17 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:18.710 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:20:18.710 INFO:teuthology.orchestra.run.trial107.stdout:{"epoch":2,"fsid":"707dbb29-0882-11f1-9262-d404e6e7d460","modified":"2026-02-13T02:20:12.541886Z","created":"2026-02-13T02:19:49.454661Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:3300","nonce":0},{"type":"v1","addr":"10.20.193.27:6789","nonce":0}]},"addr":"10.20.193.27:6789/0","public_addr":"10.20.193.27:6789/0","priority":0,"weight":0,"time_added":"2026-02-13T02:19:49:454669+0000","crush_location":[]},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:3300","nonce":0},{"type":"v1","addr":"10.20.193.107:6789","nonce":0}]},"addr":"10.20.193.107:6789/0","public_addr":"10.20.193.107:6789/0","priority":0,"weight":0,"time_added":"2026-02-13T02:20:12:541885+0000","crush_location":[]}],"quorum":[0,1]} 2026-02-13T02:20:18.710 INFO:teuthology.orchestra.run.trial107.stderr:dumped monmap epoch 2 2026-02-13T02:20:18.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:18 trial027 ceph-mon[24567]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:18.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:18 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-13T02:20:19.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:18 trial107 ceph-mon[27422]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:19.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:18 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-13T02:20:19.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:18 trial028 ceph-mon[26701]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2026-02-13T02:20:19.765 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2026-02-13T02:20:19.765 DEBUG:teuthology.orchestra.run.trial107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph mon dump -f json 2026-02-13T02:20:19.886 INFO:teuthology.orchestra.run.trial107.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.c/config 2026-02-13T02:20:24.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:23 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-13T02:20:24.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:23 trial028 ceph-mon[26701]: mon.a calling monitor election 2026-02-13T02:20:24.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:23 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-13T02:20:24.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:23 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-13T02:20:24.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:23 trial028 ceph-mon[26701]: mon.b calling monitor election 2026-02-13T02:20:24.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:23 trial028 ceph-mon[26701]: mon.c calling monitor election 2026-02-13T02:20:24.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:23 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-13T02:20:24.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:23 trial028 ceph-mon[26701]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:24.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:23 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-13T02:20:24.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:23 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-13T02:20:24.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:23 trial028 ceph-mon[26701]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:24.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:23 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-13T02:20:24.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:23 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-13T02:20:24.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:23 trial028 ceph-mon[26701]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2026-02-13T02:20:24.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:23 trial028 ceph-mon[26701]: monmap epoch 3 2026-02-13T02:20:24.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:23 trial028 ceph-mon[26701]: fsid 707dbb29-0882-11f1-9262-d404e6e7d460 2026-02-13T02:20:24.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:23 trial028 ceph-mon[26701]: last_changed 2026-02-13T02:20:18.889722+0000 2026-02-13T02:20:24.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:23 trial028 ceph-mon[26701]: created 2026-02-13T02:19:49.454661+0000 2026-02-13T02:20:24.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:23 trial028 ceph-mon[26701]: min_mon_release 20 (tentacle) 2026-02-13T02:20:24.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:23 trial028 ceph-mon[26701]: election_strategy: 1 2026-02-13T02:20:24.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:23 trial028 ceph-mon[26701]: 0: [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] mon.a 2026-02-13T02:20:24.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:23 trial028 ceph-mon[26701]: 1: [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] mon.c 2026-02-13T02:20:24.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:23 trial028 ceph-mon[26701]: 2: [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] mon.b 2026-02-13T02:20:24.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:23 trial028 ceph-mon[26701]: fsmap 2026-02-13T02:20:24.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:23 trial028 ceph-mon[26701]: osdmap e3: 0 total, 0 up, 0 in 2026-02-13T02:20:24.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:23 trial028 ceph-mon[26701]: mgrmap e8: x(active, since 27s) 2026-02-13T02:20:24.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:23 trial028 ceph-mon[26701]: overall HEALTH_OK 2026-02-13T02:20:24.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:23 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:24.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:23 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:24.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:23 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:24.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:23 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:24.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:23 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-13T02:20:24.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:23 trial107 ceph-mon[27422]: mon.a calling monitor election 2026-02-13T02:20:24.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:23 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-13T02:20:24.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:23 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-13T02:20:24.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:23 trial107 ceph-mon[27422]: mon.b calling monitor election 2026-02-13T02:20:24.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:23 trial107 ceph-mon[27422]: mon.c calling monitor election 2026-02-13T02:20:24.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:23 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-13T02:20:24.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:23 trial107 ceph-mon[27422]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:24.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:23 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-13T02:20:24.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:23 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-13T02:20:24.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:23 trial107 ceph-mon[27422]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:24.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:23 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-13T02:20:24.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:23 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-13T02:20:24.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:23 trial107 ceph-mon[27422]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2026-02-13T02:20:24.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:23 trial107 ceph-mon[27422]: monmap epoch 3 2026-02-13T02:20:24.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:23 trial107 ceph-mon[27422]: fsid 707dbb29-0882-11f1-9262-d404e6e7d460 2026-02-13T02:20:24.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:23 trial107 ceph-mon[27422]: last_changed 2026-02-13T02:20:18.889722+0000 2026-02-13T02:20:24.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:23 trial107 ceph-mon[27422]: created 2026-02-13T02:19:49.454661+0000 2026-02-13T02:20:24.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:23 trial107 ceph-mon[27422]: min_mon_release 20 (tentacle) 2026-02-13T02:20:24.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:23 trial107 ceph-mon[27422]: election_strategy: 1 2026-02-13T02:20:24.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:23 trial107 ceph-mon[27422]: 0: [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] mon.a 2026-02-13T02:20:24.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:23 trial107 ceph-mon[27422]: 1: [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] mon.c 2026-02-13T02:20:24.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:23 trial107 ceph-mon[27422]: 2: [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] mon.b 2026-02-13T02:20:24.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:23 trial107 ceph-mon[27422]: fsmap 2026-02-13T02:20:24.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:23 trial107 ceph-mon[27422]: osdmap e3: 0 total, 0 up, 0 in 2026-02-13T02:20:24.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:23 trial107 ceph-mon[27422]: mgrmap e8: x(active, since 27s) 2026-02-13T02:20:24.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:23 trial107 ceph-mon[27422]: overall HEALTH_OK 2026-02-13T02:20:24.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:23 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:24.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:23 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:24.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:23 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:24.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:23 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:24.407 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:23 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-13T02:20:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:23 trial027 ceph-mon[24567]: mon.a calling monitor election 2026-02-13T02:20:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:23 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-13T02:20:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:23 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-13T02:20:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:23 trial027 ceph-mon[24567]: mon.b calling monitor election 2026-02-13T02:20:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:23 trial027 ceph-mon[24567]: mon.c calling monitor election 2026-02-13T02:20:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:23 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-13T02:20:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:23 trial027 ceph-mon[24567]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:23 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-13T02:20:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:23 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-13T02:20:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:23 trial027 ceph-mon[24567]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:23 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-13T02:20:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:23 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-13T02:20:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:23 trial027 ceph-mon[24567]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2026-02-13T02:20:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:23 trial027 ceph-mon[24567]: monmap epoch 3 2026-02-13T02:20:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:23 trial027 ceph-mon[24567]: fsid 707dbb29-0882-11f1-9262-d404e6e7d460 2026-02-13T02:20:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:23 trial027 ceph-mon[24567]: last_changed 2026-02-13T02:20:18.889722+0000 2026-02-13T02:20:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:23 trial027 ceph-mon[24567]: created 2026-02-13T02:19:49.454661+0000 2026-02-13T02:20:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:23 trial027 ceph-mon[24567]: min_mon_release 20 (tentacle) 2026-02-13T02:20:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:23 trial027 ceph-mon[24567]: election_strategy: 1 2026-02-13T02:20:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:23 trial027 ceph-mon[24567]: 0: [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] mon.a 2026-02-13T02:20:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:23 trial027 ceph-mon[24567]: 1: [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] mon.c 2026-02-13T02:20:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:23 trial027 ceph-mon[24567]: 2: [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] mon.b 2026-02-13T02:20:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:23 trial027 ceph-mon[24567]: fsmap 2026-02-13T02:20:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:23 trial027 ceph-mon[24567]: osdmap e3: 0 total, 0 up, 0 in 2026-02-13T02:20:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:23 trial027 ceph-mon[24567]: mgrmap e8: x(active, since 27s) 2026-02-13T02:20:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:23 trial027 ceph-mon[24567]: overall HEALTH_OK 2026-02-13T02:20:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:23 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:23 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:23 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:23 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:24.548 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:20:24.548 INFO:teuthology.orchestra.run.trial107.stdout:{"epoch":3,"fsid":"707dbb29-0882-11f1-9262-d404e6e7d460","modified":"2026-02-13T02:20:18.889722Z","created":"2026-02-13T02:19:49.454661Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:3300","nonce":0},{"type":"v1","addr":"10.20.193.27:6789","nonce":0}]},"addr":"10.20.193.27:6789/0","public_addr":"10.20.193.27:6789/0","priority":0,"weight":0,"time_added":"2026-02-13T02:19:49:454669+0000","crush_location":[]},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:3300","nonce":0},{"type":"v1","addr":"10.20.193.107:6789","nonce":0}]},"addr":"10.20.193.107:6789/0","public_addr":"10.20.193.107:6789/0","priority":0,"weight":0,"time_added":"2026-02-13T02:20:12:541885+0000","crush_location":[]},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:3300","nonce":0},{"type":"v1","addr":"10.20.193.28:6789","nonce":0}]},"addr":"10.20.193.28:6789/0","public_addr":"10.20.193.28:6789/0","priority":0,"weight":0,"time_added":"2026-02-13T02:20:18:889721+0000","crush_location":[]}],"quorum":[0,1,2]} 2026-02-13T02:20:24.548 INFO:teuthology.orchestra.run.trial107.stderr:dumped monmap epoch 3 2026-02-13T02:20:24.590 INFO:tasks.cephadm:Generating final ceph.conf file... 2026-02-13T02:20:24.591 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph config generate-minimal-conf 2026-02-13T02:20:24.712 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:24.928 INFO:teuthology.orchestra.run.trial027.stdout:# minimal ceph.conf for 707dbb29-0882-11f1-9262-d404e6e7d460 2026-02-13T02:20:24.928 INFO:teuthology.orchestra.run.trial027.stdout:[global] 2026-02-13T02:20:24.928 INFO:teuthology.orchestra.run.trial027.stdout: fsid = 707dbb29-0882-11f1-9262-d404e6e7d460 2026-02-13T02:20:24.928 INFO:teuthology.orchestra.run.trial027.stdout: mon_host = [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] 2026-02-13T02:20:24.992 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2026-02-13T02:20:24.992 DEBUG:teuthology.orchestra.run.trial027:> set -ex 2026-02-13T02:20:24.992 DEBUG:teuthology.orchestra.run.trial027:> sudo dd of=/etc/ceph/ceph.conf 2026-02-13T02:20:25.018 DEBUG:teuthology.orchestra.run.trial027:> set -ex 2026-02-13T02:20:25.018 DEBUG:teuthology.orchestra.run.trial027:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2026-02-13T02:20:25.087 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-13T02:20:25.087 DEBUG:teuthology.orchestra.run.trial028:> sudo dd of=/etc/ceph/ceph.conf 2026-02-13T02:20:25.114 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-13T02:20:25.114 DEBUG:teuthology.orchestra.run.trial028:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2026-02-13T02:20:25.178 DEBUG:teuthology.orchestra.run.trial107:> set -ex 2026-02-13T02:20:25.178 DEBUG:teuthology.orchestra.run.trial107:> sudo dd of=/etc/ceph/ceph.conf 2026-02-13T02:20:25.204 DEBUG:teuthology.orchestra.run.trial107:> set -ex 2026-02-13T02:20:25.204 DEBUG:teuthology.orchestra.run.trial107:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2026-02-13T02:20:25.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:24 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:25.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:24 trial028 ceph-mon[26701]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:25.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:24 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:24 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:24 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:24 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:24 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:25.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:24 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:25.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:24 trial028 ceph-mon[26701]: from='client.? 10.20.193.107:0/1294056007' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2026-02-13T02:20:25.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:24 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:24 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:24 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:24 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:24 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:24 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:24 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:24 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2026-02-13T02:20:25.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:24 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2026-02-13T02:20:25.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:24 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:25.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:24 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-13T02:20:25.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:24 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2701263166' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:25.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:24 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:25.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:24 trial107 ceph-mon[27422]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:25.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:24 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:24 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:24 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:24 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:24 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:25.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:24 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:25.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:24 trial107 ceph-mon[27422]: from='client.? 10.20.193.107:0/1294056007' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2026-02-13T02:20:25.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:24 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:24 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:24 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:24 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:24 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:24 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:24 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.252 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:24 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2026-02-13T02:20:25.252 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:24 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2026-02-13T02:20:25.252 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:24 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:25.252 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:24 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-13T02:20:25.252 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:24 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2701263166' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:25.268 INFO:tasks.cephadm:Adding mgr.x on trial027 2026-02-13T02:20:25.268 INFO:tasks.cephadm:Adding mgr.y on trial028 2026-02-13T02:20:25.268 INFO:tasks.cephadm:Adding mgr.z on trial107 2026-02-13T02:20:25.268 DEBUG:teuthology.orchestra.run.trial107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph orch apply mgr '3;trial027=x;trial028=y;trial107=z' 2026-02-13T02:20:25.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:24 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:25.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:24 trial027 ceph-mon[24567]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:25.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:24 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:24 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:24 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:24 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:24 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:25.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:24 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:25.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:24 trial027 ceph-mon[24567]: from='client.? 10.20.193.107:0/1294056007' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2026-02-13T02:20:25.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:24 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:24 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:24 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:24 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:24 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:24 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:24 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:25.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:24 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2026-02-13T02:20:25.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:24 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2026-02-13T02:20:25.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:24 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:25.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:24 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-13T02:20:25.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:24 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2701263166' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:25.421 INFO:teuthology.orchestra.run.trial107.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.c/config 2026-02-13T02:20:25.644 INFO:teuthology.orchestra.run.trial107.stdout:Scheduled mgr update... 2026-02-13T02:20:25.686 DEBUG:teuthology.orchestra.run.trial028:mgr.y> sudo journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mgr.y.service 2026-02-13T02:20:25.689 DEBUG:teuthology.orchestra.run.trial107:mgr.z> sudo journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mgr.z.service 2026-02-13T02:20:25.692 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2026-02-13T02:20:25.692 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph config set client client_mount_timeout 600 2026-02-13T02:20:25.695 INFO:tasks.cephadm:setting config [client] debug_client = 20 2026-02-13T02:20:25.695 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph config set client debug_client 20 2026-02-13T02:20:25.739 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2026-02-13T02:20:25.740 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph config set client debug_ms 1 2026-02-13T02:20:25.783 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2026-02-13T02:20:25.783 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph config set client rados_mon_op_timeout 900 2026-02-13T02:20:25.811 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:25.812 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2026-02-13T02:20:25.812 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph config set client rados_osd_op_timeout 900 2026-02-13T02:20:25.855 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2026-02-13T02:20:25.856 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph config set mds debug_mds 20 2026-02-13T02:20:25.857 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:25.900 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2026-02-13T02:20:25.900 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph config set mds debug_mds_balancer 20 2026-02-13T02:20:25.901 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:25.930 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:25.931 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2026-02-13T02:20:25.932 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph config set mds debug_ms 2 2026-02-13T02:20:25.974 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2026-02-13T02:20:25.974 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph config set mds mds_bal_fragment_size_max 10000 2026-02-13T02:20:25.979 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:25.980 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2026-02-13T02:20:25.980 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph config set mds mds_bal_merge_size 5 2026-02-13T02:20:26.019 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:26.020 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2026-02-13T02:20:26.020 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph config set mds mds_bal_split_bits 3 2026-02-13T02:20:26.082 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:26.106 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:26.107 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2026-02-13T02:20:26.107 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph config set mds mds_bal_split_size 100 2026-02-13T02:20:26.108 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:26.142 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2026-02-13T02:20:26.142 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph config set mds mds_debug_frag True 2026-02-13T02:20:26.150 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.148+0000 7f10c39b06c0 1 -- 10.20.193.27:0/1964660631 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f109803f8f0 msgr2=0x7f109805fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.150 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.148+0000 7f10c39b06c0 1 --2- 10.20.193.27:0/1964660631 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f109803f8f0 0x7f109805fda0 secure :-1 s=READY pgs=48 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f10ac005980 tx=0x7f10ac005910 comp rx=0 tx=0).stop 2026-02-13T02:20:26.150 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.148+0000 7f10c39b06c0 1 -- 10.20.193.27:0/1964660631 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f10bc110050 msgr2=0x7f10bc171390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.150 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.148+0000 7f10c39b06c0 1 --2- 10.20.193.27:0/1964660631 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f10bc110050 0x7f10bc171390 secure :-1 s=READY pgs=88 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f10b002f680 tx=0x7f10b00046d0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.151 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.148+0000 7f10c39b06c0 1 -- 10.20.193.27:0/1964660631 shutdown_connections 2026-02-13T02:20:26.151 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.148+0000 7f10c39b06c0 1 --2- 10.20.193.27:0/1964660631 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f109803f8f0 0x7f109805fda0 unknown :-1 s=CLOSED pgs=48 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.151 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.148+0000 7f10c39b06c0 1 --2- 10.20.193.27:0/1964660631 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f10bc110a00 0x7f10bc0f91e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.151 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.148+0000 7f10c39b06c0 1 --2- 10.20.193.27:0/1964660631 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f10bc110050 0x7f10bc171390 unknown :-1 s=CLOSED pgs=88 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.151 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.148+0000 7f10c39b06c0 1 --2- 10.20.193.27:0/1964660631 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f10bc10eed0 0x7f10bc0f8980 unknown :-1 s=CLOSED pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.151 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.148+0000 7f10c39b06c0 1 -- 10.20.193.27:0/1964660631 >> 10.20.193.27:0/1964660631 conn(0x7f10bc0ec5f0 msgr2=0x7f10bc0eee20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.151 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.148+0000 7f10c39b06c0 1 -- 10.20.193.27:0/1964660631 shutdown_connections 2026-02-13T02:20:26.151 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.148+0000 7f10c39b06c0 1 -- 10.20.193.27:0/1964660631 wait complete. 2026-02-13T02:20:26.152 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:26.153 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2026-02-13T02:20:26.153 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph config set mds mds_debug_scatterstat True 2026-02-13T02:20:26.156 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.150+0000 7f078812f6c0 1 -- 10.20.193.27:0/1810926662 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0780110a00 msgr2=0x7f0780171390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.156 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.150+0000 7f078812f6c0 1 --2- 10.20.193.27:0/1810926662 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0780110a00 0x7f0780171390 secure :-1 s=READY pgs=93 gs=1 cs=0 l=1 c_cookie=5786557731297779250 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f077c00aa40 tx=0x7f077c02fde0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.156 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.150+0000 7f078812f6c0 1 -- 10.20.193.27:0/1810926662 shutdown_connections 2026-02-13T02:20:26.156 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.150+0000 7f078812f6c0 1 --2- 10.20.193.27:0/1810926662 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0780110a00 0x7f0780171390 unknown :-1 s=CLOSED pgs=93 gs=1 cs=0 l=1 c_cookie=5786557731297779250 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.156 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.150+0000 7f078812f6c0 1 --2- 10.20.193.27:0/1810926662 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f0780110050 0x7f07801104c0 unknown :-1 s=CLOSED pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.156 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.150+0000 7f078812f6c0 1 --2- 10.20.193.27:0/1810926662 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f078010eed0 0x7f078010f2c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.156 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.150+0000 7f078812f6c0 1 -- 10.20.193.27:0/1810926662 >> 10.20.193.27:0/1810926662 conn(0x7f07800ec5f0 msgr2=0x7f078010ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.156 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.150+0000 7f078812f6c0 1 -- 10.20.193.27:0/1810926662 shutdown_connections 2026-02-13T02:20:26.156 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.150+0000 7f078812f6c0 1 -- 10.20.193.27:0/1810926662 wait complete. 2026-02-13T02:20:26.156 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.151+0000 7f078812f6c0 1 Processor -- start 2026-02-13T02:20:26.157 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.151+0000 7f078812f6c0 1 -- start start 2026-02-13T02:20:26.157 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.152+0000 7f078812f6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f078010eed0 0x7f07800f8780 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.157 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.152+0000 7f078812f6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0780110050 0x7f0780171390 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.157 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.152+0000 7f078812f6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0780110a00 0x7f07800f8fe0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.157 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.152+0000 7f078812f6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f07800f9c10 con 0x7f0780110a00 2026-02-13T02:20:26.157 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.152+0000 7f078812f6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f07800f9d80 con 0x7f0780110050 2026-02-13T02:20:26.157 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.152+0000 7f078812f6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f07800f9f40 con 0x7f078010eed0 2026-02-13T02:20:26.157 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.152+0000 7f07866a16c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0780110a00 0x7f07800f8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.157 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.152+0000 7f0785ea06c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f078010eed0 0x7f07800f8780 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.157 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.152+0000 7f07866a16c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0780110a00 0x7f07800f8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:43600/0 (socket says 10.20.193.27:43600) 2026-02-13T02:20:26.157 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.152+0000 7f07866a16c0 1 -- 10.20.193.27:0/3436891875 learned_addr learned my addr 10.20.193.27:0/3436891875 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:20:26.157 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.152+0000 7f0785ea06c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f078010eed0 0x7f07800f8780 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.107:3300/0 says I am v2:10.20.193.27:41480/0 (socket says 10.20.193.27:41480) 2026-02-13T02:20:26.157 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.152+0000 7f078569f6c0 1 --2- 10.20.193.27:0/3436891875 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0780110050 0x7f0780171390 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.158 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.152+0000 7f07866a16c0 1 -- 10.20.193.27:0/3436891875 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f078010eed0 msgr2=0x7f07800f8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.158 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.152+0000 7f07866a16c0 1 --2- 10.20.193.27:0/3436891875 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f078010eed0 0x7f07800f8780 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.158 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.152+0000 7f07866a16c0 1 -- 10.20.193.27:0/3436891875 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0780110050 msgr2=0x7f0780171390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.158 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.152+0000 7f07866a16c0 1 --2- 10.20.193.27:0/3436891875 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0780110050 0x7f0780171390 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.158 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.152+0000 7f07866a16c0 1 -- 10.20.193.27:0/3436891875 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f077c00a660 con 0x7f0780110a00 2026-02-13T02:20:26.158 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.152+0000 7f07866a16c0 1 --2- 10.20.193.27:0/3436891875 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0780110a00 0x7f07800f8fe0 secure :-1 s=READY pgs=94 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f077c030990 tx=0x7f077c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.158 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.152+0000 7f076effd6c0 1 -- 10.20.193.27:0/3436891875 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f077c003d80 con 0x7f0780110a00 2026-02-13T02:20:26.158 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.152+0000 7f076effd6c0 1 -- 10.20.193.27:0/3436891875 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(25 keys) ==== 1050+0+0 (secure 0 0 0) 0x7f077c008640 con 0x7f0780110a00 2026-02-13T02:20:26.158 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.152+0000 7f078812f6c0 1 -- 10.20.193.27:0/3436891875 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07800fa1c0 con 0x7f0780110a00 2026-02-13T02:20:26.158 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.152+0000 7f078812f6c0 1 -- 10.20.193.27:0/3436891875 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07800fa640 con 0x7f0780110a00 2026-02-13T02:20:26.158 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.152+0000 7f076effd6c0 1 -- 10.20.193.27:0/3436891875 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f077c038360 con 0x7f0780110a00 2026-02-13T02:20:26.158 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.153+0000 7f076effd6c0 1 -- 10.20.193.27:0/3436891875 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 8) ==== 52814+0+0 (secure 0 0 0) 0x7f077c038530 con 0x7f0780110a00 2026-02-13T02:20:26.158 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.153+0000 7f078812f6c0 1 -- 10.20.193.27:0/3436891875 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0780175040 con 0x7f0780110a00 2026-02-13T02:20:26.159 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.153+0000 7f076effd6c0 1 --2- 10.20.193.27:0/3436891875 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f075c03f8f0 0x7f075c05fda0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.159 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.153+0000 7f076effd6c0 1 -- 10.20.193.27:0/3436891875 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 935+0+0 (secure 0 0 0) 0x7f077c076940 con 0x7f0780110a00 2026-02-13T02:20:26.159 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.154+0000 7f0785ea06c0 1 --2- 10.20.193.27:0/3436891875 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f075c03f8f0 0x7f075c05fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.159 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2026-02-13T02:20:26.160 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph config set mds mds_op_complaint_time 180 2026-02-13T02:20:26.160 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.154+0000 7f84ebf806c0 1 -- 10.20.193.27:0/2795753423 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f84e4174510 con 0x7f84e410eed0 2026-02-13T02:20:26.161 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.154+0000 7f0785ea06c0 1 --2- 10.20.193.27:0/3436891875 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f075c03f8f0 0x7f075c05fda0 secure :-1 s=READY pgs=51 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0770005980 tx=0x7f0770005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.161 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.155+0000 7f076effd6c0 1 -- 10.20.193.27:0/3436891875 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f077c038810 con 0x7f0780110a00 2026-02-13T02:20:26.162 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.158+0000 7f84d2ffd6c0 1 -- 10.20.193.27:0/2795753423 <== mon.0 v2:10.20.193.27:3300/0 10 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7f84d4005c00 con 0x7f84e410eed0 2026-02-13T02:20:26.162 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.159+0000 7f84d2ffd6c0 1 -- 10.20.193.27:0/2795753423 <== mon.0 v2:10.20.193.27:3300/0 11 ==== config(26 keys) ==== 1081+0+0 (secure 0 0 0) 0x7f84d4005c00 con 0x7f84e410eed0 2026-02-13T02:20:26.162 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.159+0000 7f076effd6c0 1 -- 10.20.193.27:0/3436891875 <== mon.0 v2:10.20.193.27:3300/0 7 ==== config(26 keys) ==== 1081+0+0 (secure 0 0 0) 0x7f077c076370 con 0x7f0780110a00 2026-02-13T02:20:26.162 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.159+0000 7f820e7fc6c0 1 -- 10.20.193.27:0/4160888720 <== mon.0 v2:10.20.193.27:3300/0 8 ==== config(26 keys) ==== 1081+0+0 (secure 0 0 0) 0x7f8214010030 con 0x7f8220110050 2026-02-13T02:20:26.163 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2026-02-13T02:20:26.163 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph config set mds mds_verify_scatter True 2026-02-13T02:20:26.164 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.162+0000 7f84ebf806c0 1 -- 10.20.193.27:0/2795753423 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f84c003f8f0 msgr2=0x7f84c005fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.165 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.162+0000 7f84ebf806c0 1 --2- 10.20.193.27:0/2795753423 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f84c003f8f0 0x7f84c005fda0 secure :-1 s=READY pgs=49 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f84d8004750 tx=0x7f84d8005f10 comp rx=0 tx=0).stop 2026-02-13T02:20:26.165 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.162+0000 7f84ebf806c0 1 -- 10.20.193.27:0/2795753423 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f84e410eed0 msgr2=0x7f84e40f88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.165 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.162+0000 7f84ebf806c0 1 --2- 10.20.193.27:0/2795753423 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f84e410eed0 0x7f84e40f88a0 secure :-1 s=READY pgs=90 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f84d402f680 tx=0x7f84d40046d0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.165 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.162+0000 7f84ebf806c0 1 -- 10.20.193.27:0/2795753423 shutdown_connections 2026-02-13T02:20:26.165 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.162+0000 7f84ebf806c0 1 --2- 10.20.193.27:0/2795753423 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f84c003f8f0 0x7f84c005fda0 unknown :-1 s=CLOSED pgs=49 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.165 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.162+0000 7f84ebf806c0 1 --2- 10.20.193.27:0/2795753423 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f84e4110a00 0x7f84e40f9100 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.165 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.162+0000 7f84ebf806c0 1 --2- 10.20.193.27:0/2795753423 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f84e4110050 0x7f84e4171390 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.165 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.162+0000 7f84ebf806c0 1 --2- 10.20.193.27:0/2795753423 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f84e410eed0 0x7f84e40f88a0 unknown :-1 s=CLOSED pgs=90 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.165 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.162+0000 7f84ebf806c0 1 -- 10.20.193.27:0/2795753423 >> 10.20.193.27:0/2795753423 conn(0x7f84e40ec5f0 msgr2=0x7f84e40eedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.165 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.162+0000 7f84ebf806c0 1 -- 10.20.193.27:0/2795753423 shutdown_connections 2026-02-13T02:20:26.166 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.162+0000 7f84ebf806c0 1 -- 10.20.193.27:0/2795753423 wait complete. 2026-02-13T02:20:26.204 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.202+0000 7febe8f006c0 1 -- 10.20.193.27:0/3988979097 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7febe410e760 msgr2=0x7febe416f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.204 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.202+0000 7febe8f006c0 1 --2- 10.20.193.27:0/3988979097 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7febe410e760 0x7febe416f0f0 secure :-1 s=READY pgs=95 gs=1 cs=0 l=1 c_cookie=4119069450658298042 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7febd400aa40 tx=0x7febd402fde0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.204 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.202+0000 7febe8f006c0 1 -- 10.20.193.27:0/3988979097 shutdown_connections 2026-02-13T02:20:26.204 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.202+0000 7febe8f006c0 1 --2- 10.20.193.27:0/3988979097 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7febe410e760 0x7febe416f0f0 unknown :-1 s=CLOSED pgs=95 gs=1 cs=0 l=1 c_cookie=4119069450658298042 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.204 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.202+0000 7febe8f006c0 1 --2- 10.20.193.27:0/3988979097 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7febe410ddb0 0x7febe410e220 unknown :-1 s=CLOSED pgs=0 gs=5 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.205 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.202+0000 7febe8f006c0 1 --2- 10.20.193.27:0/3988979097 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7febe410cc30 0x7febe410d020 unknown :-1 s=CLOSED pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.205 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.202+0000 7febe8f006c0 1 -- 10.20.193.27:0/3988979097 >> 10.20.193.27:0/3988979097 conn(0x7febe40ea330 msgr2=0x7febe410a760 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.205 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.202+0000 7febe8f006c0 1 -- 10.20.193.27:0/3988979097 shutdown_connections 2026-02-13T02:20:26.205 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.202+0000 7febe8f006c0 1 -- 10.20.193.27:0/3988979097 wait complete. 2026-02-13T02:20:26.206 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2026-02-13T02:20:26.206 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph config set mds osd_op_complaint_time 180 2026-02-13T02:20:26.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.203+0000 7febe8f006c0 1 Processor -- start 2026-02-13T02:20:26.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.203+0000 7febe8f006c0 1 -- start start 2026-02-13T02:20:26.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.203+0000 7febe8f006c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7febe410cc30 0x7febe40f63b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.203+0000 7febe8f006c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7febe410ddb0 0x7febe416f0f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.203+0000 7febe8f006c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7febe410e760 0x7febe40f6c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.203+0000 7febe8f006c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7febe40f7840 con 0x7febe410e760 2026-02-13T02:20:26.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.203+0000 7febe8f006c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7febe40f79b0 con 0x7febe410ddb0 2026-02-13T02:20:26.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.203+0000 7febe8f006c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7febe40f7b70 con 0x7febe410cc30 2026-02-13T02:20:26.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.203+0000 7febe2d766c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7febe410e760 0x7febe40f6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.203+0000 7febe2d766c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7febe410e760 0x7febe40f6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:43604/0 (socket says 10.20.193.27:43604) 2026-02-13T02:20:26.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.203+0000 7febe2d766c0 1 -- 10.20.193.27:0/1595880533 learned_addr learned my addr 10.20.193.27:0/1595880533 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:20:26.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.203+0000 7febe1d746c0 1 --2- 10.20.193.27:0/1595880533 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7febe410ddb0 0x7febe416f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.203+0000 7febe25756c0 1 --2- 10.20.193.27:0/1595880533 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7febe410cc30 0x7febe40f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.203+0000 7febe2d766c0 1 -- 10.20.193.27:0/1595880533 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7febe410cc30 msgr2=0x7febe40f63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.203+0000 7febe2d766c0 1 --2- 10.20.193.27:0/1595880533 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7febe410cc30 0x7febe40f63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.203+0000 7febe2d766c0 1 -- 10.20.193.27:0/1595880533 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7febe410ddb0 msgr2=0x7febe416f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.203+0000 7febe2d766c0 1 --2- 10.20.193.27:0/1595880533 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7febe410ddb0 0x7febe416f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.203+0000 7febe2d766c0 1 -- 10.20.193.27:0/1595880533 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd400a660 con 0x7febe410e760 2026-02-13T02:20:26.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.204+0000 7febe2d766c0 1 --2- 10.20.193.27:0/1595880533 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7febe410e760 0x7febe40f6c10 secure :-1 s=READY pgs=96 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7febd4006ec0 tx=0x7febd40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.204+0000 7febcb7fe6c0 1 -- 10.20.193.27:0/1595880533 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7febd4003d00 con 0x7febe410e760 2026-02-13T02:20:26.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.204+0000 7febcb7fe6c0 1 -- 10.20.193.27:0/1595880533 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(26 keys) ==== 1081+0+0 (secure 0 0 0) 0x7febd4003e60 con 0x7febe410e760 2026-02-13T02:20:26.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.204+0000 7febe8f006c0 1 -- 10.20.193.27:0/1595880533 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe40f7df0 con 0x7febe410e760 2026-02-13T02:20:26.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.204+0000 7febcb7fe6c0 1 -- 10.20.193.27:0/1595880533 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7febd40382f0 con 0x7febe410e760 2026-02-13T02:20:26.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.204+0000 7febe8f006c0 1 -- 10.20.193.27:0/1595880533 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febe40f8270 con 0x7febe410e760 2026-02-13T02:20:26.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.205+0000 7febcb7fe6c0 1 -- 10.20.193.27:0/1595880533 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 8) ==== 52814+0+0 (secure 0 0 0) 0x7febd40081c0 con 0x7febe410e760 2026-02-13T02:20:26.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.205+0000 7febe8f006c0 1 -- 10.20.193.27:0/1595880533 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febe4172eb0 con 0x7febe410e760 2026-02-13T02:20:26.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.205+0000 7febcb7fe6c0 1 --2- 10.20.193.27:0/1595880533 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7febbc03f8f0 0x7febbc05fda0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.205+0000 7febcb7fe6c0 1 -- 10.20.193.27:0/1595880533 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 935+0+0 (secure 0 0 0) 0x7febd4076830 con 0x7febe410e760 2026-02-13T02:20:26.211 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.206+0000 7febe25756c0 1 --2- 10.20.193.27:0/1595880533 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7febbc03f8f0 0x7febbc05fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.212 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.206+0000 7febe25756c0 1 --2- 10.20.193.27:0/1595880533 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7febbc03f8f0 0x7febbc05fda0 secure :-1 s=READY pgs=52 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7febd8005980 tx=0x7febd8005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.212 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.207+0000 7febcb7fe6c0 1 -- 10.20.193.27:0/1595880533 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7febd4008960 con 0x7febe410e760 2026-02-13T02:20:26.213 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2026-02-13T02:20:26.213 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph config set mds rados_mon_op_timeout 900 2026-02-13T02:20:26.217 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:26 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:26.217 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:26 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:26.218 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:26 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2026-02-13T02:20:26.218 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:26 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2026-02-13T02:20:26.218 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:26 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:26.218 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:26 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:26.218 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:26 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:26.218 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:26 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2026-02-13T02:20:26.218 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:26 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2026-02-13T02:20:26.218 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:26 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:26.218 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:26 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:26.218 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:26 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:26.218 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:26 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:26.218 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:26 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:26.219 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:26 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:26.219 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:26 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:26.219 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:26 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:26.219 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:26 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2026-02-13T02:20:26.219 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:26 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2026-02-13T02:20:26.219 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:26 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2026-02-13T02:20:26.219 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:26 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:26.219 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:26 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/803294146' entity='client.admin' 2026-02-13T02:20:26.219 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:26 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/4032541679' entity='client.admin' 2026-02-13T02:20:26.221 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2026-02-13T02:20:26.222 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph config set mds rados_osd_op_timeout 900 2026-02-13T02:20:26.225 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.223+0000 7f82274786c0 1 -- 10.20.193.27:0/4160888720 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f82201745f0 con 0x7f8220110050 2026-02-13T02:20:26.226 INFO:tasks.cephadm:setting config [mgr] client_mount_timeout = 30 2026-02-13T02:20:26.226 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph config set mgr client_mount_timeout 30 2026-02-13T02:20:26.228 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.226+0000 7f820e7fc6c0 1 -- 10.20.193.27:0/4160888720 <== mon.0 v2:10.20.193.27:3300/0 9 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v13)=0 v13) ==== 123+0+0 (secure 0 0 0) 0x7f821400e8d0 con 0x7f8220110050 2026-02-13T02:20:26.229 INFO:tasks.cephadm:setting config [mgr] debug_client = 20 2026-02-13T02:20:26.229 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph config set mgr debug_client 20 2026-02-13T02:20:26.230 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.228+0000 7f820e7fc6c0 1 -- 10.20.193.27:0/4160888720 <== mon.0 v2:10.20.193.27:3300/0 10 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f821400e8d0 con 0x7f8220110050 2026-02-13T02:20:26.231 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.228+0000 7f076effd6c0 1 -- 10.20.193.27:0/3436891875 <== mon.0 v2:10.20.193.27:3300/0 8 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f077c03b510 con 0x7f0780110a00 2026-02-13T02:20:26.231 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.227+0000 7febcb7fe6c0 1 -- 10.20.193.27:0/1595880533 <== mon.0 v2:10.20.193.27:3300/0 7 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7febd4076240 con 0x7febe410e760 2026-02-13T02:20:26.232 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.230+0000 7f82274786c0 1 -- 10.20.193.27:0/4160888720 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f81fc03f8f0 msgr2=0x7f81fc05fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.232 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.230+0000 7f82274786c0 1 --2- 10.20.193.27:0/4160888720 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f81fc03f8f0 0x7f81fc05fda0 secure :-1 s=READY pgs=50 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f82100098c0 tx=0x7f82100023a0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.233 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.230+0000 7f82274786c0 1 -- 10.20.193.27:0/4160888720 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8220110050 msgr2=0x7f8220171390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.233 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.230+0000 7f82274786c0 1 --2- 10.20.193.27:0/4160888720 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8220110050 0x7f8220171390 secure :-1 s=READY pgs=92 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8214005dc0 tx=0x7f821400bbd0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.233 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.230+0000 7f82274786c0 1 -- 10.20.193.27:0/4160888720 shutdown_connections 2026-02-13T02:20:26.233 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.230+0000 7f82274786c0 1 --2- 10.20.193.27:0/4160888720 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f81fc03f8f0 0x7f81fc05fda0 unknown :-1 s=CLOSED pgs=50 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.233 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.230+0000 7f82274786c0 1 --2- 10.20.193.27:0/4160888720 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8220110a00 0x7f82200f90c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.233 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.230+0000 7f82274786c0 1 --2- 10.20.193.27:0/4160888720 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8220110050 0x7f8220171390 unknown :-1 s=CLOSED pgs=92 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.233 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.230+0000 7f82274786c0 1 --2- 10.20.193.27:0/4160888720 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f822010eed0 0x7f82200f8860 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.233 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.230+0000 7f82274786c0 1 -- 10.20.193.27:0/4160888720 >> 10.20.193.27:0/4160888720 conn(0x7f82200ec5f0 msgr2=0x7f82200eedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.233 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.230+0000 7f82274786c0 1 -- 10.20.193.27:0/4160888720 shutdown_connections 2026-02-13T02:20:26.233 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.230+0000 7f82274786c0 1 -- 10.20.193.27:0/4160888720 wait complete. 2026-02-13T02:20:26.234 INFO:tasks.cephadm:setting config [mgr] debug_mgr = 20 2026-02-13T02:20:26.234 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph config set mgr debug_mgr 20 2026-02-13T02:20:26.246 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:26.248 INFO:tasks.cephadm:setting config [mgr] debug_ms = 1 2026-02-13T02:20:26.248 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph config set mgr debug_ms 1 2026-02-13T02:20:26.252 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.251+0000 7f5359d2c6c0 1 -- 10.20.193.27:0/4074429693 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f535410cc30 msgr2=0x7f535410d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.253 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.251+0000 7f5359d2c6c0 1 --2- 10.20.193.27:0/4074429693 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f535410cc30 0x7f535410d020 secure :-1 s=READY pgs=97 gs=1 cs=0 l=1 c_cookie=16070985952034959974 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5340009970 tx=0x7f534002ed20 comp rx=0 tx=0).stop 2026-02-13T02:20:26.254 INFO:tasks.cephadm:setting config [mgr] mon_warn_on_pool_no_app = False 2026-02-13T02:20:26.254 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph config set mgr mon_warn_on_pool_no_app False 2026-02-13T02:20:26.257 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.251+0000 7f5359d2c6c0 1 -- 10.20.193.27:0/4074429693 shutdown_connections 2026-02-13T02:20:26.257 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.251+0000 7f5359d2c6c0 1 --2- 10.20.193.27:0/4074429693 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f535410e760 0x7f535416f0f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.257 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.251+0000 7f5359d2c6c0 1 --2- 10.20.193.27:0/4074429693 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f535410ddb0 0x7f535410e220 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.257 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.251+0000 7f5359d2c6c0 1 --2- 10.20.193.27:0/4074429693 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f535410cc30 0x7f535410d020 unknown :-1 s=CLOSED pgs=97 gs=1 cs=0 l=1 c_cookie=16070985952034959974 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.257 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.251+0000 7f5359d2c6c0 1 -- 10.20.193.27:0/4074429693 >> 10.20.193.27:0/4074429693 conn(0x7f53540ea330 msgr2=0x7f535410a760 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.257 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.251+0000 7f5359d2c6c0 1 -- 10.20.193.27:0/4074429693 shutdown_connections 2026-02-13T02:20:26.258 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.251+0000 7f5359d2c6c0 1 -- 10.20.193.27:0/4074429693 wait complete. 2026-02-13T02:20:26.258 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.252+0000 7f5359d2c6c0 1 Processor -- start 2026-02-13T02:20:26.258 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.252+0000 7f5359d2c6c0 1 -- start start 2026-02-13T02:20:26.258 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.252+0000 7f5359d2c6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f535410cc30 0x7f53540f8650 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.258 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.252+0000 7f5359d2c6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f535410ddb0 0x7f535416f0f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.258 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.252+0000 7f5359d2c6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f535410e760 0x7f53540f8eb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.258 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.252+0000 7f5352ffd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f535410ddb0 0x7f535416f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.258 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.252+0000 7f5359d2c6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f53540f9ae0 con 0x7f535410ddb0 2026-02-13T02:20:26.258 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.252+0000 7f5359d2c6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f53540f9c50 con 0x7f535410cc30 2026-02-13T02:20:26.258 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.252+0000 7f5359d2c6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f53540f9e10 con 0x7f535410e760 2026-02-13T02:20:26.258 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.252+0000 7f5352ffd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f535410ddb0 0x7f535416f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:43630/0 (socket says 10.20.193.27:43630) 2026-02-13T02:20:26.259 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.252+0000 7f5352ffd6c0 1 -- 10.20.193.27:0/1874777830 learned_addr learned my addr 10.20.193.27:0/1874777830 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:20:26.259 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.253+0000 7f5353fff6c0 1 --2- 10.20.193.27:0/1874777830 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f535410e760 0x7f53540f8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.259 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.253+0000 7f5352ffd6c0 1 -- 10.20.193.27:0/1874777830 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f535410e760 msgr2=0x7f53540f8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.259 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.253+0000 7f5352ffd6c0 1 --2- 10.20.193.27:0/1874777830 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f535410e760 0x7f53540f8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.259 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.253+0000 7f5352ffd6c0 1 -- 10.20.193.27:0/1874777830 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f535410cc30 msgr2=0x7f53540f8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.259 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.253+0000 7f53537fe6c0 1 --2- 10.20.193.27:0/1874777830 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f535410cc30 0x7f53540f8650 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.259 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.253+0000 7f5352ffd6c0 1 --2- 10.20.193.27:0/1874777830 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f535410cc30 0x7f53540f8650 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.259 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.253+0000 7f5352ffd6c0 1 -- 10.20.193.27:0/1874777830 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5340009590 con 0x7f535410ddb0 2026-02-13T02:20:26.259 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.253+0000 7f5353fff6c0 1 --2- 10.20.193.27:0/1874777830 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f535410e760 0x7f53540f8eb0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:20:26.259 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.253+0000 7f53537fe6c0 1 --2- 10.20.193.27:0/1874777830 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f535410cc30 0x7f53540f8650 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:20:26.259 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.253+0000 7f5352ffd6c0 1 --2- 10.20.193.27:0/1874777830 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f535410ddb0 0x7f535416f0f0 secure :-1 s=READY pgs=98 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f534400b710 tx=0x7f534400bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.259 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.253+0000 7f5350ff96c0 1 -- 10.20.193.27:0/1874777830 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5344004250 con 0x7f535410ddb0 2026-02-13T02:20:26.260 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.254+0000 7f5350ff96c0 1 -- 10.20.193.27:0/1874777830 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5344002d80 con 0x7f535410ddb0 2026-02-13T02:20:26.260 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.254+0000 7f5359d2c6c0 1 -- 10.20.193.27:0/1874777830 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53540fa0f0 con 0x7f535410ddb0 2026-02-13T02:20:26.260 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.254+0000 7f5359d2c6c0 1 -- 10.20.193.27:0/1874777830 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53540fa620 con 0x7f535410ddb0 2026-02-13T02:20:26.260 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.254+0000 7f5350ff96c0 1 -- 10.20.193.27:0/1874777830 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f534400cdc0 con 0x7f535410ddb0 2026-02-13T02:20:26.261 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2026-02-13T02:20:26.261 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph config set mon mon_op_complaint_time 120 2026-02-13T02:20:26.262 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.255+0000 7f5350ff96c0 1 -- 10.20.193.27:0/1874777830 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 8) ==== 52814+0+0 (secure 0 0 0) 0x7f53440071c0 con 0x7f535410ddb0 2026-02-13T02:20:26.262 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.255+0000 7f5359d2c6c0 1 -- 10.20.193.27:0/1874777830 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5354173020 con 0x7f535410ddb0 2026-02-13T02:20:26.262 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.255+0000 7f5350ff96c0 1 --2- 10.20.193.27:0/1874777830 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f532803f8f0 0x7f532805fda0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.263 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.256+0000 7f5350ff96c0 1 -- 10.20.193.27:0/1874777830 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 935+0+0 (secure 0 0 0) 0x7f5344052360 con 0x7f535410ddb0 2026-02-13T02:20:26.263 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.256+0000 7f53537fe6c0 1 --2- 10.20.193.27:0/1874777830 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f532803f8f0 0x7f532805fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.263 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.257+0000 7f53537fe6c0 1 --2- 10.20.193.27:0/1874777830 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f532803f8f0 0x7f532805fda0 secure :-1 s=READY pgs=53 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f53400098c0 tx=0x7f5340009230 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.263 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.259+0000 7f5350ff96c0 1 -- 10.20.193.27:0/1874777830 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f53440153f0 con 0x7f535410ddb0 2026-02-13T02:20:26.263 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.261+0000 7f629bc176c0 1 -- 10.20.193.27:0/950906388 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f629410ddd0 msgr2=0x7f629410e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.263 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.261+0000 7f629bc176c0 1 --2- 10.20.193.27:0/950906388 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f629410ddd0 0x7f629410e240 secure :-1 s=READY pgs=99 gs=2 cs=0 l=1 c_cookie=7466603574524445552 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6288009b00 tx=0x7f628802eeb0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.264 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.261+0000 7f629bc176c0 1 -- 10.20.193.27:0/950906388 shutdown_connections 2026-02-13T02:20:26.264 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.261+0000 7f629bc176c0 1 --2- 10.20.193.27:0/950906388 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f629410e780 0x7f629416f110 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.264 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.261+0000 7f629bc176c0 1 --2- 10.20.193.27:0/950906388 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f629410ddd0 0x7f629410e240 unknown :-1 s=CLOSED pgs=99 gs=2 cs=0 l=1 c_cookie=7466603574524445552 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.264 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.261+0000 7f629bc176c0 1 --2- 10.20.193.27:0/950906388 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f629410cc50 0x7f629410d040 unknown :-1 s=CLOSED pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.264 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.261+0000 7f629bc176c0 1 -- 10.20.193.27:0/950906388 >> 10.20.193.27:0/950906388 conn(0x7f62940ea370 msgr2=0x7f629410a780 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.264 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.261+0000 7f629bc176c0 1 -- 10.20.193.27:0/950906388 shutdown_connections 2026-02-13T02:20:26.264 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.261+0000 7f629bc176c0 1 -- 10.20.193.27:0/950906388 wait complete. 2026-02-13T02:20:26.267 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2026-02-13T02:20:26.267 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph config set osd osd_op_complaint_time 180 2026-02-13T02:20:26.268 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.262+0000 7f629bc176c0 1 Processor -- start 2026-02-13T02:20:26.269 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.262+0000 7f629bc176c0 1 -- start start 2026-02-13T02:20:26.269 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.262+0000 7f629bc176c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f629410cc50 0x7f62940f6380 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.269 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.263+0000 7f629bc176c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f629410ddd0 0x7f629416f0f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.269 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.263+0000 7f629bc176c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f629410e780 0x7f62940f6bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.269 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.263+0000 7f629a1896c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f629410e780 0x7f62940f6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.269 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.263+0000 7f629a1896c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f629410e780 0x7f62940f6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:43640/0 (socket says 10.20.193.27:43640) 2026-02-13T02:20:26.269 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.263+0000 7f629a1896c0 1 -- 10.20.193.27:0/318736707 learned_addr learned my addr 10.20.193.27:0/318736707 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:20:26.269 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.263+0000 7f629bc176c0 1 -- 10.20.193.27:0/318736707 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f62940f7810 con 0x7f629410e780 2026-02-13T02:20:26.269 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.263+0000 7f629bc176c0 1 -- 10.20.193.27:0/318736707 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f62940f7980 con 0x7f629410ddd0 2026-02-13T02:20:26.269 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.263+0000 7f629bc176c0 1 -- 10.20.193.27:0/318736707 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f62940f7b40 con 0x7f629410cc50 2026-02-13T02:20:26.270 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.263+0000 7f629a1896c0 1 -- 10.20.193.27:0/318736707 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f629410cc50 msgr2=0x7f62940f6380 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2026-02-13T02:20:26.270 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.263+0000 7f629a1896c0 1 --2- 10.20.193.27:0/318736707 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f629410cc50 0x7f62940f6380 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.270 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.263+0000 7f629a1896c0 1 -- 10.20.193.27:0/318736707 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f629410ddd0 msgr2=0x7f629416f0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2026-02-13T02:20:26.270 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.263+0000 7f629a1896c0 1 --2- 10.20.193.27:0/318736707 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f629410ddd0 0x7f629416f0f0 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.270 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.263+0000 7f629a1896c0 1 -- 10.20.193.27:0/318736707 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62880094c0 con 0x7f629410e780 2026-02-13T02:20:26.270 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.263+0000 7f629a1896c0 1 --2- 10.20.193.27:0/318736707 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f629410e780 0x7f62940f6bc0 secure :-1 s=READY pgs=100 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f629000c6c0 tx=0x7f629000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.270 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.264+0000 7f6282ffd6c0 1 -- 10.20.193.27:0/318736707 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6290003a30 con 0x7f629410e780 2026-02-13T02:20:26.270 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.264+0000 7f6282ffd6c0 1 -- 10.20.193.27:0/318736707 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f62900054e0 con 0x7f629410e780 2026-02-13T02:20:26.270 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.264+0000 7f6282ffd6c0 1 -- 10.20.193.27:0/318736707 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f629000dc50 con 0x7f629410e780 2026-02-13T02:20:26.270 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.264+0000 7f629bc176c0 1 -- 10.20.193.27:0/318736707 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62940f7dc0 con 0x7f629410e780 2026-02-13T02:20:26.270 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.264+0000 7f629bc176c0 1 -- 10.20.193.27:0/318736707 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62940f81f0 con 0x7f629410e780 2026-02-13T02:20:26.270 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.265+0000 7f6282ffd6c0 1 -- 10.20.193.27:0/318736707 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 8) ==== 52814+0+0 (secure 0 0 0) 0x7f6290013070 con 0x7f629410e780 2026-02-13T02:20:26.271 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.265+0000 7f6282ffd6c0 1 --2- 10.20.193.27:0/318736707 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f6270043e50 0x7f6270064300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.271 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.265+0000 7f6282ffd6c0 1 -- 10.20.193.27:0/318736707 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 935+0+0 (secure 0 0 0) 0x7f6290056480 con 0x7f629410e780 2026-02-13T02:20:26.271 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.265+0000 7f629bc176c0 1 -- 10.20.193.27:0/318736707 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6294173070 con 0x7f629410e780 2026-02-13T02:20:26.271 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.267+0000 7f62999886c0 1 --2- 10.20.193.27:0/318736707 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f6270043e50 0x7f6270064300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.284 DEBUG:tasks.cephadm:set 26 configs 2026-02-13T02:20:26.285 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '707dbb29-0882-11f1-9262-d404e6e7d460', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='trial027') 2026-02-13T02:20:26.285 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '707dbb29-0882-11f1-9262-d404e6e7d460', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='trial027') 2026-02-13T02:20:26.285 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '707dbb29-0882-11f1-9262-d404e6e7d460', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='trial027') 2026-02-13T02:20:26.285 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '707dbb29-0882-11f1-9262-d404e6e7d460', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='trial027') 2026-02-13T02:20:26.285 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '707dbb29-0882-11f1-9262-d404e6e7d460', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='trial027') 2026-02-13T02:20:26.286 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.267+0000 7f6282ffd6c0 1 -- 10.20.193.27:0/318736707 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f6290013bb0 con 0x7f629410e780 2026-02-13T02:20:26.286 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.269+0000 7f62999886c0 1 --2- 10.20.193.27:0/318736707 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f6270043e50 0x7f6270064300 secure :-1 s=READY pgs=54 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6284005b40 tx=0x7f6284005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.294 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:26.297 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.294+0000 7f078812f6c0 1 -- 10.20.193.27:0/3436891875 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f07801747d0 con 0x7f0780110a00 2026-02-13T02:20:26.299 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.298+0000 7f076effd6c0 1 -- 10.20.193.27:0/3436891875 <== mon.0 v2:10.20.193.27:3300/0 9 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v14)=0 v14) ==== 108+0+0 (secure 0 0 0) 0x7f077c0338d0 con 0x7f0780110a00 2026-02-13T02:20:26.305 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.303+0000 7f078812f6c0 1 -- 10.20.193.27:0/3436891875 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f075c03f8f0 msgr2=0x7f075c05fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.305 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.303+0000 7f078812f6c0 1 --2- 10.20.193.27:0/3436891875 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f075c03f8f0 0x7f075c05fda0 secure :-1 s=READY pgs=51 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0770005980 tx=0x7f0770005910 comp rx=0 tx=0).stop 2026-02-13T02:20:26.305 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.303+0000 7f078812f6c0 1 -- 10.20.193.27:0/3436891875 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0780110a00 msgr2=0x7f07800f8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.305 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.303+0000 7f078812f6c0 1 --2- 10.20.193.27:0/3436891875 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0780110a00 0x7f07800f8fe0 secure :-1 s=READY pgs=94 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f077c030990 tx=0x7f077c005800 comp rx=0 tx=0).stop 2026-02-13T02:20:26.307 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.306+0000 7f078812f6c0 1 -- 10.20.193.27:0/3436891875 shutdown_connections 2026-02-13T02:20:26.307 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.306+0000 7f078812f6c0 1 --2- 10.20.193.27:0/3436891875 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f075c03f8f0 0x7f075c05fda0 unknown :-1 s=CLOSED pgs=51 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.306+0000 7f078812f6c0 1 --2- 10.20.193.27:0/3436891875 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0780110a00 0x7f07800f8fe0 unknown :-1 s=CLOSED pgs=94 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.306+0000 7f078812f6c0 1 --2- 10.20.193.27:0/3436891875 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0780110050 0x7f0780171390 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.306+0000 7f078812f6c0 1 --2- 10.20.193.27:0/3436891875 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f078010eed0 0x7f07800f8780 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.306+0000 7f078812f6c0 1 -- 10.20.193.27:0/3436891875 >> 10.20.193.27:0/3436891875 conn(0x7f07800ec5f0 msgr2=0x7f07800eed00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.310 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.308+0000 7f078812f6c0 1 -- 10.20.193.27:0/3436891875 shutdown_connections 2026-02-13T02:20:26.310 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.308+0000 7f078812f6c0 1 -- 10.20.193.27:0/3436891875 wait complete. 2026-02-13T02:20:26.312 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '707dbb29-0882-11f1-9262-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='trial027') 2026-02-13T02:20:26.328 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:26.328 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:26.331 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.329+0000 7fd53a3db6c0 1 -- 10.20.193.27:0/1878425325 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd534110a00 msgr2=0x7fd534171390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.333 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.329+0000 7fd53a3db6c0 1 --2- 10.20.193.27:0/1878425325 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd534110a00 0x7fd534171390 secure :-1 s=READY pgs=101 gs=4 cs=0 l=1 c_cookie=1236705110032439160 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd528006fd0 tx=0x7fd52802fd20 comp rx=0 tx=0).stop 2026-02-13T02:20:26.333 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.329+0000 7fd53a3db6c0 1 -- 10.20.193.27:0/1878425325 shutdown_connections 2026-02-13T02:20:26.333 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.329+0000 7fd53a3db6c0 1 --2- 10.20.193.27:0/1878425325 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd534110a00 0x7fd534171390 unknown :-1 s=CLOSED pgs=101 gs=4 cs=0 l=1 c_cookie=1236705110032439160 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.334 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.329+0000 7fd53a3db6c0 1 --2- 10.20.193.27:0/1878425325 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd534110050 0x7fd5341104c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.334 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.329+0000 7fd53a3db6c0 1 --2- 10.20.193.27:0/1878425325 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd53410eed0 0x7fd53410f2c0 unknown :-1 s=CLOSED pgs=0 gs=5 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.334 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.329+0000 7fd53a3db6c0 1 -- 10.20.193.27:0/1878425325 >> 10.20.193.27:0/1878425325 conn(0x7fd5340ec5f0 msgr2=0x7fd53410ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.334 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.329+0000 7fd53a3db6c0 1 -- 10.20.193.27:0/1878425325 shutdown_connections 2026-02-13T02:20:26.334 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.329+0000 7fd53a3db6c0 1 -- 10.20.193.27:0/1878425325 wait complete. 2026-02-13T02:20:26.334 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.332+0000 7fd53a3db6c0 1 Processor -- start 2026-02-13T02:20:26.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.332+0000 7fd53a3db6c0 1 -- start start 2026-02-13T02:20:26.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.332+0000 7fd53a3db6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd53410eed0 0x7fd5340f8550 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.332+0000 7fd53a3db6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd534110050 0x7fd534171390 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.332+0000 7fd53a3db6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd534110a00 0x7fd5340f8db0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.332+0000 7fd53a3db6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd5340f99e0 con 0x7fd534110a00 2026-02-13T02:20:26.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.332+0000 7fd53a3db6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd5340f9b50 con 0x7fd53410eed0 2026-02-13T02:20:26.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.332+0000 7fd53a3db6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd5340f9d10 con 0x7fd534110050 2026-02-13T02:20:26.336 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:26.336 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.332+0000 7fd533fff6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd53410eed0 0x7fd5340f8550 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.336 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.332+0000 7fd5337fe6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd534110050 0x7fd534171390 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.337 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.332+0000 7fd533fff6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd53410eed0 0x7fd5340f8550 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.27:43098/0 (socket says 10.20.193.27:43098) 2026-02-13T02:20:26.337 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.332+0000 7fd533fff6c0 1 -- 10.20.193.27:0/2920323902 learned_addr learned my addr 10.20.193.27:0/2920323902 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:20:26.337 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.333+0000 7fd533fff6c0 1 -- 10.20.193.27:0/2920323902 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd53410eed0 msgr2=0x7fd5340f8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 9 2026-02-13T02:20:26.337 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.333+0000 7fd533fff6c0 1 -- 10.20.193.27:0/2920323902 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd53410eed0 msgr2=0x7fd5340f8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-13T02:20:26.337 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.333+0000 7fd533fff6c0 1 --2- 10.20.193.27:0/2920323902 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd53410eed0 0x7fd5340f8550 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-13T02:20:26.337 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.333+0000 7fd533fff6c0 1 --2- 10.20.193.27:0/2920323902 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd53410eed0 0x7fd5340f8550 unknown :-1 s=START_CONNECT pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-13T02:20:26.337 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.333+0000 7fd5337fe6c0 1 -- 10.20.193.27:0/2920323902 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd534110050 msgr2=0x7fd534171390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 10 2026-02-13T02:20:26.337 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.333+0000 7fd5337fe6c0 1 -- 10.20.193.27:0/2920323902 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd534110050 msgr2=0x7fd534171390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-13T02:20:26.337 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.333+0000 7fd5337fe6c0 1 --2- 10.20.193.27:0/2920323902 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd534110050 0x7fd534171390 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-13T02:20:26.337 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.333+0000 7fd5337fe6c0 1 --2- 10.20.193.27:0/2920323902 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd534110050 0x7fd534171390 unknown :-1 s=START_CONNECT pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-13T02:20:26.337 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.333+0000 7fd53894d6c0 1 --2- 10.20.193.27:0/2920323902 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd534110a00 0x7fd5340f8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.338 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.333+0000 7fd53894d6c0 1 -- 10.20.193.27:0/2920323902 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd534110050 msgr2=0x7fd534171390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-13T02:20:26.338 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.333+0000 7fd53894d6c0 1 --2- 10.20.193.27:0/2920323902 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd534110050 0x7fd534171390 unknown :-1 s=START_CONNECT pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.338 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.333+0000 7fd53894d6c0 1 -- 10.20.193.27:0/2920323902 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd53410eed0 msgr2=0x7fd5340f8550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-13T02:20:26.338 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.333+0000 7fd53894d6c0 1 --2- 10.20.193.27:0/2920323902 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd53410eed0 0x7fd5340f8550 unknown :-1 s=START_CONNECT pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.338 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.333+0000 7fd53894d6c0 1 -- 10.20.193.27:0/2920323902 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd52800a5b0 con 0x7fd534110a00 2026-02-13T02:20:26.338 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.333+0000 7fd53894d6c0 1 --2- 10.20.193.27:0/2920323902 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd534110a00 0x7fd5340f8db0 secure :-1 s=READY pgs=102 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd528006fd0 tx=0x7fd528003a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.338 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.334+0000 7fd5317fa6c0 1 -- 10.20.193.27:0/2920323902 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd528003d50 con 0x7fd534110a00 2026-02-13T02:20:26.338 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.334+0000 7fd5317fa6c0 1 -- 10.20.193.27:0/2920323902 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd528008680 con 0x7fd534110a00 2026-02-13T02:20:26.338 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.334+0000 7fd53a3db6c0 1 -- 10.20.193.27:0/2920323902 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5340f9f90 con 0x7fd534110a00 2026-02-13T02:20:26.339 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.335+0000 7fd53a3db6c0 1 -- 10.20.193.27:0/2920323902 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5340fa410 con 0x7fd534110a00 2026-02-13T02:20:26.339 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.335+0000 7fd5317fa6c0 1 -- 10.20.193.27:0/2920323902 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd528037520 con 0x7fd534110a00 2026-02-13T02:20:26.339 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.335+0000 7fd5317fa6c0 1 -- 10.20.193.27:0/2920323902 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 8) ==== 52814+0+0 (secure 0 0 0) 0x7fd5280376b0 con 0x7fd534110a00 2026-02-13T02:20:26.339 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.336+0000 7fd5317fa6c0 1 --2- 10.20.193.27:0/2920323902 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd50803f8a0 0x7fd50805fd50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.339 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.336+0000 7fd5317fa6c0 1 -- 10.20.193.27:0/2920323902 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 935+0+0 (secure 0 0 0) 0x7fd52803a030 con 0x7fd534110a00 2026-02-13T02:20:26.339 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.336+0000 7fd533fff6c0 1 --2- 10.20.193.27:0/2920323902 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd50803f8a0 0x7fd50805fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=6 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.342 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.336+0000 7fd506ffd6c0 1 -- 10.20.193.27:0/2920323902 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5341752e0 con 0x7fd534110a00 2026-02-13T02:20:26.342 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.340+0000 7fd5317fa6c0 1 -- 10.20.193.27:0/2920323902 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fd528031920 con 0x7fd534110a00 2026-02-13T02:20:26.342 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.340+0000 7fd533fff6c0 1 --2- 10.20.193.27:0/2920323902 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd50803f8a0 0x7fd50805fd50 secure :-1 s=READY pgs=55 gs=6 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd520007240 tx=0x7fd520007620 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.383 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:26.385 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.383+0000 7febe8f006c0 1 -- 10.20.193.27:0/1595880533 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7febe4172550 con 0x7febe410e760 2026-02-13T02:20:26.385 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '707dbb29-0882-11f1-9262-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='trial027') 2026-02-13T02:20:26.388 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:26.389 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.388+0000 7febcb7fe6c0 1 -- 10.20.193.27:0/1595880533 <== mon.0 v2:10.20.193.27:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v15)=0 v15) ==== 117+0+0 (secure 0 0 0) 0x7febd403a8c0 con 0x7febe410e760 2026-02-13T02:20:26.395 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.393+0000 7febe8f006c0 1 -- 10.20.193.27:0/1595880533 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7febbc03f8f0 msgr2=0x7febbc05fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.395 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.393+0000 7febe8f006c0 1 --2- 10.20.193.27:0/1595880533 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7febbc03f8f0 0x7febbc05fda0 secure :-1 s=READY pgs=52 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7febd8005980 tx=0x7febd8005910 comp rx=0 tx=0).stop 2026-02-13T02:20:26.395 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.393+0000 7febe8f006c0 1 -- 10.20.193.27:0/1595880533 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7febe410e760 msgr2=0x7febe40f6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.395 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.393+0000 7febe8f006c0 1 --2- 10.20.193.27:0/1595880533 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7febe410e760 0x7febe40f6c10 secure :-1 s=READY pgs=96 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7febd4006ec0 tx=0x7febd40058a0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.397 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.395+0000 7febe8f006c0 1 -- 10.20.193.27:0/1595880533 shutdown_connections 2026-02-13T02:20:26.397 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.395+0000 7febe8f006c0 1 --2- 10.20.193.27:0/1595880533 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7febbc03f8f0 0x7febbc05fda0 unknown :-1 s=CLOSED pgs=52 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.397 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.395+0000 7febe8f006c0 1 --2- 10.20.193.27:0/1595880533 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7febe410e760 0x7febe40f6c10 unknown :-1 s=CLOSED pgs=96 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.397 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.395+0000 7febe8f006c0 1 --2- 10.20.193.27:0/1595880533 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7febe410ddb0 0x7febe416f0f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.397 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.395+0000 7febe8f006c0 1 --2- 10.20.193.27:0/1595880533 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7febe410cc30 0x7febe40f63b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.397 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.395+0000 7febe8f006c0 1 -- 10.20.193.27:0/1595880533 >> 10.20.193.27:0/1595880533 conn(0x7febe40ea330 msgr2=0x7febe40ec970 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.398 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.397+0000 7febe8f006c0 1 -- 10.20.193.27:0/1595880533 shutdown_connections 2026-02-13T02:20:26.398 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.397+0000 7febe8f006c0 1 -- 10.20.193.27:0/1595880533 wait complete. 2026-02-13T02:20:26.415 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:26.416 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.414+0000 7f629bc176c0 1 -- 10.20.193.27:0/318736707 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7f629410d040 con 0x7f629410e780 2026-02-13T02:20:26.419 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.417+0000 7f6282ffd6c0 1 -- 10.20.193.27:0/318736707 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v16)=0 v16) ==== 127+0+0 (secure 0 0 0) 0x7f6290014df0 con 0x7f629410e780 2026-02-13T02:20:26.421 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:26.422 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:26.423 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.421+0000 7f629bc176c0 1 -- 10.20.193.27:0/318736707 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f6270043e50 msgr2=0x7f6270064300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.423 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.421+0000 7f629bc176c0 1 --2- 10.20.193.27:0/318736707 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f6270043e50 0x7f6270064300 secure :-1 s=READY pgs=54 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6284005b40 tx=0x7f6284005910 comp rx=0 tx=0).stop 2026-02-13T02:20:26.423 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.421+0000 7f629bc176c0 1 -- 10.20.193.27:0/318736707 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f629410e780 msgr2=0x7f62940f6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.423 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.421+0000 7f629bc176c0 1 --2- 10.20.193.27:0/318736707 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f629410e780 0x7f62940f6bc0 secure :-1 s=READY pgs=100 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f629000c6c0 tx=0x7f629000cb90 comp rx=0 tx=0).stop 2026-02-13T02:20:26.430 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.429+0000 7f629bc176c0 1 -- 10.20.193.27:0/318736707 shutdown_connections 2026-02-13T02:20:26.431 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.429+0000 7f629bc176c0 1 --2- 10.20.193.27:0/318736707 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f6270043e50 0x7f6270064300 unknown :-1 s=CLOSED pgs=54 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.431 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.429+0000 7f629bc176c0 1 --2- 10.20.193.27:0/318736707 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f629410e780 0x7f62940f6bc0 unknown :-1 s=CLOSED pgs=100 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.431 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.429+0000 7f629bc176c0 1 --2- 10.20.193.27:0/318736707 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f629410ddd0 0x7f629416f0f0 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.431 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.429+0000 7f629bc176c0 1 --2- 10.20.193.27:0/318736707 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f629410cc50 0x7f62940f6380 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.431 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.429+0000 7f629bc176c0 1 -- 10.20.193.27:0/318736707 >> 10.20.193.27:0/318736707 conn(0x7f62940ea370 msgr2=0x7f62940ec930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.432 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.430+0000 7f629bc176c0 1 -- 10.20.193.27:0/318736707 shutdown_connections 2026-02-13T02:20:26.432 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.430+0000 7f629bc176c0 1 -- 10.20.193.27:0/318736707 wait complete. 2026-02-13T02:20:26.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.432+0000 7f5359d2c6c0 1 -- 10.20.193.27:0/1874777830 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f53540fa960 con 0x7f535410ddb0 2026-02-13T02:20:26.438 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.429+0000 7fb8ebf536c0 1 -- 10.20.193.27:0/2577534352 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb8e4110a00 msgr2=0x7fb8e4171390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.438 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.429+0000 7fb8ebf536c0 1 --2- 10.20.193.27:0/2577534352 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb8e4110a00 0x7fb8e4171390 secure :-1 s=READY pgs=103 gs=1 cs=0 l=1 c_cookie=15405791385911143128 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb8e000aa40 tx=0x7fb8e002fe00 comp rx=0 tx=0).stop 2026-02-13T02:20:26.438 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.431+0000 7fb8ebf536c0 1 -- 10.20.193.27:0/2577534352 shutdown_connections 2026-02-13T02:20:26.438 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.431+0000 7fb8ebf536c0 1 --2- 10.20.193.27:0/2577534352 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb8e4110a00 0x7fb8e4171390 unknown :-1 s=CLOSED pgs=103 gs=1 cs=0 l=1 c_cookie=15405791385911143128 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.438 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.431+0000 7fb8ebf536c0 1 --2- 10.20.193.27:0/2577534352 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb8e4110050 0x7fb8e41104c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.438 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.431+0000 7fb8ebf536c0 1 --2- 10.20.193.27:0/2577534352 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb8e410eed0 0x7fb8e410f2c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.438 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.431+0000 7fb8ebf536c0 1 -- 10.20.193.27:0/2577534352 >> 10.20.193.27:0/2577534352 conn(0x7fb8e40ec5f0 msgr2=0x7fb8e410ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.438 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.431+0000 7fb8ebf536c0 1 -- 10.20.193.27:0/2577534352 shutdown_connections 2026-02-13T02:20:26.438 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.431+0000 7fb8ebf536c0 1 -- 10.20.193.27:0/2577534352 wait complete. 2026-02-13T02:20:26.438 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.431+0000 7fb8ebf536c0 1 Processor -- start 2026-02-13T02:20:26.439 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.431+0000 7fb8ebf536c0 1 -- start start 2026-02-13T02:20:26.439 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.432+0000 7fb8ebf536c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb8e410eed0 0x7fb8e40f8580 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.439 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.432+0000 7fb8ebf536c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb8e4110050 0x7fb8e4171390 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.439 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.432+0000 7fb8ebf536c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb8e4110a00 0x7fb8e40f8de0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.439 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.432+0000 7fb8ebf536c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e40f9a10 con 0x7fb8e410eed0 2026-02-13T02:20:26.439 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.432+0000 7fb8ebf536c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e40f9b80 con 0x7fb8e4110050 2026-02-13T02:20:26.439 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.432+0000 7fb8ebf536c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e40f9d40 con 0x7fb8e4110a00 2026-02-13T02:20:26.439 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.432+0000 7fb8e94c36c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb8e4110050 0x7fb8e4171390 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.439 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.432+0000 7fb8ea4c56c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb8e4110a00 0x7fb8e40f8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.439 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.432+0000 7fb8e94c36c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb8e4110050 0x7fb8e4171390 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.27:43118/0 (socket says 10.20.193.27:43118) 2026-02-13T02:20:26.439 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.432+0000 7fb8e94c36c0 1 -- 10.20.193.27:0/2673424342 learned_addr learned my addr 10.20.193.27:0/2673424342 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:20:26.440 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.432+0000 7fb8e94c36c0 1 -- 10.20.193.27:0/2673424342 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb8e4110050 msgr2=0x7fb8e4171390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 9 2026-02-13T02:20:26.440 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.432+0000 7fb8e94c36c0 1 -- 10.20.193.27:0/2673424342 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb8e4110050 msgr2=0x7fb8e4171390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-13T02:20:26.440 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.432+0000 7fb8e94c36c0 1 --2- 10.20.193.27:0/2673424342 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb8e4110050 0x7fb8e4171390 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-13T02:20:26.440 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.432+0000 7fb8e94c36c0 1 --2- 10.20.193.27:0/2673424342 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb8e4110050 0x7fb8e4171390 unknown :-1 s=START_CONNECT pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-13T02:20:26.440 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.432+0000 7fb8e9cc46c0 1 --2- 10.20.193.27:0/2673424342 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb8e410eed0 0x7fb8e40f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.440 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.433+0000 7fb8e9cc46c0 1 -- 10.20.193.27:0/2673424342 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb8e4110a00 msgr2=0x7fb8e40f8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.440 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.433+0000 7fb8e9cc46c0 1 --2- 10.20.193.27:0/2673424342 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb8e4110a00 0x7fb8e40f8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.440 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.433+0000 7fb8e9cc46c0 1 -- 10.20.193.27:0/2673424342 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb8e4110050 msgr2=0x7fb8e4171390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-13T02:20:26.440 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.433+0000 7fb8e9cc46c0 1 --2- 10.20.193.27:0/2673424342 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb8e4110050 0x7fb8e4171390 unknown :-1 s=START_CONNECT pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.440 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.433+0000 7fb8e9cc46c0 1 -- 10.20.193.27:0/2673424342 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e000a660 con 0x7fb8e410eed0 2026-02-13T02:20:26.440 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.433+0000 7fb8e9cc46c0 1 --2- 10.20.193.27:0/2673424342 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb8e410eed0 0x7fb8e40f8580 secure :-1 s=READY pgs=104 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb8d400b440 tx=0x7fb8d400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.440 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.433+0000 7fb8d2ffd6c0 1 -- 10.20.193.27:0/2673424342 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb8d40044a0 con 0x7fb8e410eed0 2026-02-13T02:20:26.441 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.433+0000 7fb8d2ffd6c0 1 -- 10.20.193.27:0/2673424342 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb8d4002b70 con 0x7fb8e410eed0 2026-02-13T02:20:26.441 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.433+0000 7fb8d2ffd6c0 1 -- 10.20.193.27:0/2673424342 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb8d400cbd0 con 0x7fb8e410eed0 2026-02-13T02:20:26.441 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.433+0000 7fb8ebf536c0 1 -- 10.20.193.27:0/2673424342 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e40f9fc0 con 0x7fb8e410eed0 2026-02-13T02:20:26.441 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.433+0000 7fb8ebf536c0 1 -- 10.20.193.27:0/2673424342 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e40fa450 con 0x7fb8e410eed0 2026-02-13T02:20:26.441 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.435+0000 7fb8ebf536c0 1 -- 10.20.193.27:0/2673424342 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8e4175320 con 0x7fb8e410eed0 2026-02-13T02:20:26.441 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.435+0000 7fb8d2ffd6c0 1 -- 10.20.193.27:0/2673424342 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 8) ==== 52814+0+0 (secure 0 0 0) 0x7fb8d400e070 con 0x7fb8e410eed0 2026-02-13T02:20:26.441 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.435+0000 7fb8d2ffd6c0 1 --2- 10.20.193.27:0/2673424342 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb8c003f8f0 0x7fb8c005fda0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.442 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:26 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:26.442 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:26 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:26.442 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:26 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2026-02-13T02:20:26.442 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:26 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2026-02-13T02:20:26.442 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:26 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:26.442 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:26 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:26.443 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:26 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:26.443 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:26 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2026-02-13T02:20:26.443 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:26 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2026-02-13T02:20:26.443 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:26 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:26.443 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:26 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:26.443 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:26 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:26.443 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:26 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:26.443 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:26 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:26.443 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:26 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:26.443 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:26 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:26.443 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:26 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:26.443 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:26 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2026-02-13T02:20:26.443 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:26 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2026-02-13T02:20:26.444 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:26 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2026-02-13T02:20:26.444 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:26 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:26.444 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:26 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/803294146' entity='client.admin' 2026-02-13T02:20:26.444 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:26 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/4032541679' entity='client.admin' 2026-02-13T02:20:26.444 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.437+0000 7f5350ff96c0 1 -- 10.20.193.27:0/1874777830 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v17)=0 v17) ==== 106+0+0 (secure 0 0 0) 0x7f5344012030 con 0x7f535410ddb0 2026-02-13T02:20:26.444 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:26.445 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.438+0000 7fb8d2ffd6c0 1 -- 10.20.193.27:0/2673424342 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 935+0+0 (secure 0 0 0) 0x7fb8d40138a0 con 0x7fb8e410eed0 2026-02-13T02:20:26.445 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.438+0000 7fb8e94c36c0 1 --2- 10.20.193.27:0/2673424342 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb8c003f8f0 0x7fb8c005fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.445 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.438+0000 7fb8e94c36c0 1 --2- 10.20.193.27:0/2673424342 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb8c003f8f0 0x7fb8c005fda0 secure :-1 s=READY pgs=56 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb8d8009260 tx=0x7fb8d8009050 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.445 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:26.446 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.441+0000 7fb8d2ffd6c0 1 -- 10.20.193.27:0/2673424342 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fb8d4012a20 con 0x7fb8e410eed0 2026-02-13T02:20:26.446 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.442+0000 7f5359d2c6c0 1 -- 10.20.193.27:0/1874777830 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f532803f8f0 msgr2=0x7f532805fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.446 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.442+0000 7f5359d2c6c0 1 --2- 10.20.193.27:0/1874777830 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f532803f8f0 0x7f532805fda0 secure :-1 s=READY pgs=53 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f53400098c0 tx=0x7f5340009230 comp rx=0 tx=0).stop 2026-02-13T02:20:26.446 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.442+0000 7f5359d2c6c0 1 -- 10.20.193.27:0/1874777830 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f535410ddb0 msgr2=0x7f535416f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.446 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.442+0000 7f5359d2c6c0 1 --2- 10.20.193.27:0/1874777830 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f535410ddb0 0x7f535416f0f0 secure :-1 s=READY pgs=98 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f534400b710 tx=0x7f534400bbe0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.446 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.443+0000 7f5359d2c6c0 1 -- 10.20.193.27:0/1874777830 shutdown_connections 2026-02-13T02:20:26.446 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.443+0000 7f5359d2c6c0 1 --2- 10.20.193.27:0/1874777830 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f532803f8f0 0x7f532805fda0 unknown :-1 s=CLOSED pgs=53 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.447 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.443+0000 7f5359d2c6c0 1 --2- 10.20.193.27:0/1874777830 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f535410e760 0x7f53540f8eb0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.447 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.443+0000 7f5359d2c6c0 1 --2- 10.20.193.27:0/1874777830 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f535410ddb0 0x7f535416f0f0 unknown :-1 s=CLOSED pgs=98 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.447 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.443+0000 7f5359d2c6c0 1 --2- 10.20.193.27:0/1874777830 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f535410cc30 0x7f53540f8650 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.447 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.443+0000 7f5359d2c6c0 1 -- 10.20.193.27:0/1874777830 >> 10.20.193.27:0/1874777830 conn(0x7f53540ea330 msgr2=0x7f5354060e10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.447 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.443+0000 7f5359d2c6c0 1 -- 10.20.193.27:0/1874777830 shutdown_connections 2026-02-13T02:20:26.447 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.443+0000 7f5359d2c6c0 1 -- 10.20.193.27:0/1874777830 wait complete. 2026-02-13T02:20:26.447 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:26.473 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:26.479 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:26.483 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '707dbb29-0882-11f1-9262-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='trial027') 2026-02-13T02:20:26.492 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.490+0000 7fd506ffd6c0 1 -- 10.20.193.27:0/2920323902 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7fd534174980 con 0x7fd534110a00 2026-02-13T02:20:26.493 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.491+0000 7f859aa516c0 1 -- 10.20.193.27:0/4086574397 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8594110050 msgr2=0x7f85941104c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.493 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.491+0000 7f859aa516c0 1 --2- 10.20.193.27:0/4086574397 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8594110050 0x7f85941104c0 secure :-1 s=READY pgs=106 gs=1 cs=0 l=1 c_cookie=11877450991149119098 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8584013ec0 tx=0x7f8584037770 comp rx=0 tx=0).stop 2026-02-13T02:20:26.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:26 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:26.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:26 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:26.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:26 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2026-02-13T02:20:26.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:26 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2026-02-13T02:20:26.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:26 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:26.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:26 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:26.495 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:26 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:26.495 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:26 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2026-02-13T02:20:26.495 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:26 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2026-02-13T02:20:26.495 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:26 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:26.495 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:26 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:26.495 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:26 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:26.495 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:26 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:26.495 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:26 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:26.495 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:26 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:26.495 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:26 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:26.495 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:26 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:26.496 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:26 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2026-02-13T02:20:26.496 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:26 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2026-02-13T02:20:26.496 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:26 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2026-02-13T02:20:26.496 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:26 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:26.496 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:26 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/803294146' entity='client.admin' 2026-02-13T02:20:26.496 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:26 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/4032541679' entity='client.admin' 2026-02-13T02:20:26.497 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.492+0000 7f859aa516c0 1 -- 10.20.193.27:0/4086574397 shutdown_connections 2026-02-13T02:20:26.497 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.492+0000 7f859aa516c0 1 --2- 10.20.193.27:0/4086574397 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8594110a00 0x7f8594171390 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.497 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.492+0000 7f859aa516c0 1 --2- 10.20.193.27:0/4086574397 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8594110050 0x7f85941104c0 unknown :-1 s=CLOSED pgs=106 gs=1 cs=0 l=1 c_cookie=11877450991149119098 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.497 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.492+0000 7f859aa516c0 1 --2- 10.20.193.27:0/4086574397 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f859410eed0 0x7f859410f2c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.497 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.492+0000 7f859aa516c0 1 -- 10.20.193.27:0/4086574397 >> 10.20.193.27:0/4086574397 conn(0x7f85940ec5f0 msgr2=0x7f859410ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.497 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.492+0000 7f859aa516c0 1 -- 10.20.193.27:0/4086574397 shutdown_connections 2026-02-13T02:20:26.498 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.492+0000 7f859aa516c0 1 -- 10.20.193.27:0/4086574397 wait complete. 2026-02-13T02:20:26.498 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.492+0000 7f859aa516c0 1 Processor -- start 2026-02-13T02:20:26.498 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.493+0000 7f859aa516c0 1 -- start start 2026-02-13T02:20:26.498 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.493+0000 7f859aa516c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f859410eed0 0x7f8594100b00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.498 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.493+0000 7f859aa516c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8594110050 0x7f8594171390 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.498 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.493+0000 7f859aa516c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8594110a00 0x7f8594101360 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.498 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.493+0000 7f859aa516c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8594101f90 con 0x7f8594110050 2026-02-13T02:20:26.498 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.493+0000 7f859aa516c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8594102100 con 0x7f859410eed0 2026-02-13T02:20:26.498 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.493+0000 7f859aa516c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f85941022c0 con 0x7f8594110a00 2026-02-13T02:20:26.498 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.493+0000 7f85937fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8594110050 0x7f8594171390 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.498 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.493+0000 7f85937fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8594110050 0x7f8594171390 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:43728/0 (socket says 10.20.193.27:43728) 2026-02-13T02:20:26.499 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.493+0000 7f85937fe6c0 1 -- 10.20.193.27:0/4073212428 learned_addr learned my addr 10.20.193.27:0/4073212428 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:20:26.499 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.493+0000 7f8593fff6c0 1 --2- 10.20.193.27:0/4073212428 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f859410eed0 0x7f8594100b00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.499 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.493+0000 7f85937fe6c0 1 -- 10.20.193.27:0/4073212428 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8594110a00 msgr2=0x7f8594101360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.499 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.493+0000 7f85937fe6c0 1 --2- 10.20.193.27:0/4073212428 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8594110a00 0x7f8594101360 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.499 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.493+0000 7fd5317fa6c0 1 -- 10.20.193.27:0/2920323902 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v18)=0 v18) ==== 116+0+0 (secure 0 0 0) 0x7fd528033920 con 0x7fd534110a00 2026-02-13T02:20:26.500 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.491+0000 7f48cb7fe6c0 1 -- 10.20.193.27:0/2213198349 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f48c402fd60 con 0x7f48d410eed0 2026-02-13T02:20:26.500 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.491+0000 7f48dac416c0 1 -- 10.20.193.27:0/2213198349 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f48d410eed0 msgr2=0x7f48d410f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.500 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.491+0000 7f48dac416c0 1 --2- 10.20.193.27:0/2213198349 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f48d410eed0 0x7f48d410f2c0 secure :-1 s=READY pgs=105 gs=1 cs=0 l=1 c_cookie=13021754499664414893 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f48c4009970 tx=0x7f48c402ed20 comp rx=0 tx=0).stop 2026-02-13T02:20:26.500 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.491+0000 7f48dac416c0 1 -- 10.20.193.27:0/2213198349 shutdown_connections 2026-02-13T02:20:26.501 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.491+0000 7f48dac416c0 1 --2- 10.20.193.27:0/2213198349 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f48d4110a00 0x7f48d4171390 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.501 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.491+0000 7f48dac416c0 1 --2- 10.20.193.27:0/2213198349 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f48d4110050 0x7f48d41104c0 unknown :-1 s=CLOSED pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.501 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.491+0000 7f48dac416c0 1 --2- 10.20.193.27:0/2213198349 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f48d410eed0 0x7f48d410f2c0 unknown :-1 s=CLOSED pgs=105 gs=1 cs=0 l=1 c_cookie=13021754499664414893 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.501 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.491+0000 7f48dac416c0 1 -- 10.20.193.27:0/2213198349 >> 10.20.193.27:0/2213198349 conn(0x7f48d40ec5f0 msgr2=0x7f48d410ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.501 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.491+0000 7f48dac416c0 1 -- 10.20.193.27:0/2213198349 shutdown_connections 2026-02-13T02:20:26.501 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.491+0000 7f48dac416c0 1 -- 10.20.193.27:0/2213198349 wait complete. 2026-02-13T02:20:26.501 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.491+0000 7f48dac416c0 1 Processor -- start 2026-02-13T02:20:26.501 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.492+0000 7f48dac416c0 1 -- start start 2026-02-13T02:20:26.501 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.492+0000 7f48dac416c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f48d410eed0 0x7f48d411ab00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.501 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.492+0000 7f48dac416c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f48d4110050 0x7f48d411b150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.501 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.492+0000 7f48dac416c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f48d4110a00 0x7f48d411bd40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.502 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.492+0000 7f48dac416c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f48d411c4d0 con 0x7f48d4110050 2026-02-13T02:20:26.502 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.492+0000 7f48dac416c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f48d411c640 con 0x7f48d410eed0 2026-02-13T02:20:26.502 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.492+0000 7f48cbfff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f48d4110050 0x7f48d411b150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.502 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.492+0000 7f48dac416c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f48d411c800 con 0x7f48d4110a00 2026-02-13T02:20:26.502 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.492+0000 7f48cbfff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f48d4110050 0x7f48d411b150 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:43724/0 (socket says 10.20.193.27:43724) 2026-02-13T02:20:26.502 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.492+0000 7f48cbfff6c0 1 -- 10.20.193.27:0/2718718715 learned_addr learned my addr 10.20.193.27:0/2718718715 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:20:26.502 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.492+0000 7f48d89b26c0 1 --2- 10.20.193.27:0/2718718715 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f48d410eed0 0x7f48d411ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.502 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.492+0000 7f48d91b36c0 1 --2- 10.20.193.27:0/2718718715 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f48d4110a00 0x7f48d411bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.502 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.492+0000 7f48cbfff6c0 1 -- 10.20.193.27:0/2718718715 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f48d4110a00 msgr2=0x7f48d411bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.502 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.492+0000 7f48cbfff6c0 1 --2- 10.20.193.27:0/2718718715 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f48d4110a00 0x7f48d411bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.502 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.492+0000 7f48d89b26c0 1 -- 10.20.193.27:0/2718718715 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f48d410eed0 msgr2=0x7f48d411ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 9 2026-02-13T02:20:26.502 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.492+0000 7f48d89b26c0 1 -- 10.20.193.27:0/2718718715 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f48d410eed0 msgr2=0x7f48d411ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-13T02:20:26.503 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.492+0000 7f48cbfff6c0 1 -- 10.20.193.27:0/2718718715 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f48d410eed0 msgr2=0x7f48d411ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.503 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.492+0000 7f48d89b26c0 1 --2- 10.20.193.27:0/2718718715 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f48d410eed0 0x7f48d411ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-13T02:20:26.503 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.492+0000 7f48d89b26c0 1 --2- 10.20.193.27:0/2718718715 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f48d410eed0 0x7f48d411ab00 unknown :-1 s=START_CONNECT pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-13T02:20:26.503 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.492+0000 7f48cbfff6c0 1 --2- 10.20.193.27:0/2718718715 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f48d410eed0 0x7f48d411ab00 unknown :-1 s=START_CONNECT pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.503 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.492+0000 7f48cbfff6c0 1 -- 10.20.193.27:0/2718718715 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c4009590 con 0x7f48d4110050 2026-02-13T02:20:26.503 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.492+0000 7f48cbfff6c0 1 --2- 10.20.193.27:0/2718718715 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f48d4110050 0x7f48d411b150 secure :-1 s=READY pgs=108 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f48bc005dc0 tx=0x7f48bc00bbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.503 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.493+0000 7f48c9ffb6c0 1 -- 10.20.193.27:0/2718718715 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f48bc004250 con 0x7f48d4110050 2026-02-13T02:20:26.503 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.493+0000 7f48dac416c0 1 -- 10.20.193.27:0/2718718715 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48d411cae0 con 0x7f48d4110050 2026-02-13T02:20:26.503 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.493+0000 7f48dac416c0 1 -- 10.20.193.27:0/2718718715 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48d415d030 con 0x7f48d4110050 2026-02-13T02:20:26.503 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.494+0000 7f48c9ffb6c0 1 -- 10.20.193.27:0/2718718715 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f48bc00be30 con 0x7f48d4110050 2026-02-13T02:20:26.504 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.493+0000 7f85937fe6c0 1 -- 10.20.193.27:0/4073212428 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f859410eed0 msgr2=0x7f8594100b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.504 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.493+0000 7f85937fe6c0 1 --2- 10.20.193.27:0/4073212428 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f859410eed0 0x7f8594100b00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.504 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.493+0000 7f85937fe6c0 1 -- 10.20.193.27:0/4073212428 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8584013ae0 con 0x7f8594110050 2026-02-13T02:20:26.504 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.493+0000 7f85937fe6c0 1 --2- 10.20.193.27:0/4073212428 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8594110050 0x7f8594171390 secure :-1 s=READY pgs=109 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8584005b00 tx=0x7f8584003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.504 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.494+0000 7f85917fa6c0 1 -- 10.20.193.27:0/4073212428 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8584003ef0 con 0x7f8594110050 2026-02-13T02:20:26.504 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.494+0000 7f85917fa6c0 1 -- 10.20.193.27:0/4073212428 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f8584009030 con 0x7f8594110050 2026-02-13T02:20:26.505 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.494+0000 7f85917fa6c0 1 -- 10.20.193.27:0/4073212428 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8584007440 con 0x7f8594110050 2026-02-13T02:20:26.505 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.494+0000 7f859aa516c0 1 -- 10.20.193.27:0/4073212428 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8594102540 con 0x7f8594110050 2026-02-13T02:20:26.505 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.494+0000 7f48c9ffb6c0 1 -- 10.20.193.27:0/2718718715 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f48bc00dea0 con 0x7f48d4110050 2026-02-13T02:20:26.505 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.494+0000 7f48a37fe6c0 1 -- 10.20.193.27:0/2718718715 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4894005310 con 0x7f48d4110050 2026-02-13T02:20:26.505 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.494+0000 7f48c9ffb6c0 1 -- 10.20.193.27:0/2718718715 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 8) ==== 52814+0+0 (secure 0 0 0) 0x7f48bc0028c0 con 0x7f48d4110050 2026-02-13T02:20:26.506 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.494+0000 7f859aa516c0 1 -- 10.20.193.27:0/4073212428 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85941028b0 con 0x7f8594110050 2026-02-13T02:20:26.506 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.495+0000 7f48c9ffb6c0 1 --2- 10.20.193.27:0/2718718715 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f48b003f8e0 0x7f48b005fd90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.506 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.495+0000 7f48c9ffb6c0 1 -- 10.20.193.27:0/2718718715 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 935+0+0 (secure 0 0 0) 0x7f48bc0512d0 con 0x7f48d4110050 2026-02-13T02:20:26.507 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.494+0000 7f18695736c0 1 -- 10.20.193.27:0/696662209 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1854002d70 con 0x7f186c0554b0 2026-02-13T02:20:26.507 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.494+0000 7f1870a986c0 1 -- 10.20.193.27:0/696662209 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f186c0554b0 msgr2=0x7f186c10c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.507 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.494+0000 7f1870a986c0 1 --2- 10.20.193.27:0/696662209 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f186c0554b0 0x7f186c10c840 secure :-1 s=READY pgs=107 gs=1 cs=0 l=1 c_cookie=10018961711645150746 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f185400aac0 tx=0x7f185402fea0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.507 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.494+0000 7f1870a986c0 1 -- 10.20.193.27:0/696662209 shutdown_connections 2026-02-13T02:20:26.507 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.494+0000 7f1870a986c0 1 --2- 10.20.193.27:0/696662209 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f186c14d2c0 0x7f186c16d6a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.507 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.494+0000 7f1870a986c0 1 --2- 10.20.193.27:0/696662209 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f186c0554b0 0x7f186c10c840 unknown :-1 s=CLOSED pgs=107 gs=1 cs=0 l=1 c_cookie=10018961711645150746 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.507 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.494+0000 7f1870a986c0 1 --2- 10.20.193.27:0/696662209 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f186c10ae10 0x7f186c10b200 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.507 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.494+0000 7f1870a986c0 1 -- 10.20.193.27:0/696662209 >> 10.20.193.27:0/696662209 conn(0x7f186c0e8510 msgr2=0x7f186c108940 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.508 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.495+0000 7f48d89b26c0 1 --2- 10.20.193.27:0/2718718715 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f48b003f8e0 0x7f48b005fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.508 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.495+0000 7f1870a986c0 1 -- 10.20.193.27:0/696662209 shutdown_connections 2026-02-13T02:20:26.508 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.495+0000 7f85917fa6c0 1 -- 10.20.193.27:0/4073212428 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 8) ==== 52814+0+0 (secure 0 0 0) 0x7f8584004050 con 0x7f8594110050 2026-02-13T02:20:26.509 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.495+0000 7f48d89b26c0 1 --2- 10.20.193.27:0/2718718715 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f48b003f8e0 0x7f48b005fd90 secure :-1 s=READY pgs=57 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f48c40023e0 tx=0x7f48c402ff90 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.509 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.495+0000 7f1870a986c0 1 -- 10.20.193.27:0/696662209 wait complete. 2026-02-13T02:20:26.509 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.495+0000 7f1870a986c0 1 Processor -- start 2026-02-13T02:20:26.510 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.495+0000 7f1870a986c0 1 -- start start 2026-02-13T02:20:26.510 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.495+0000 7f1870a986c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f186c0554b0 0x7f186c0ef2b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.510 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7f186a5756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f186c0554b0 0x7f186c0ef2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.510 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7f186a5756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f186c0554b0 0x7f186c0ef2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:43736/0 (socket says 10.20.193.27:43736) 2026-02-13T02:20:26.510 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7f186a5756c0 1 -- 10.20.193.27:0/2232626932 learned_addr learned my addr 10.20.193.27:0/2232626932 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:20:26.510 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7f85917fa6c0 1 --2- 10.20.193.27:0/4073212428 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f856803f940 0x7f856805fdf0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.511 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7f1870a986c0 1 --2- 10.20.193.27:0/2232626932 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f186c10ae10 0x7f186c0ef900 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.511 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7f1870a986c0 1 --2- 10.20.193.27:0/2232626932 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f186c14d2c0 0x7f186c0f04f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.511 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7f1870a986c0 1 -- 10.20.193.27:0/2232626932 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f186c0f0c80 con 0x7f186c0554b0 2026-02-13T02:20:26.511 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7f1870a986c0 1 -- 10.20.193.27:0/2232626932 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f186c0f0df0 con 0x7f186c14d2c0 2026-02-13T02:20:26.511 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7f1870a986c0 1 -- 10.20.193.27:0/2232626932 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f186c0f0fb0 con 0x7f186c10ae10 2026-02-13T02:20:26.511 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7f1869d746c0 1 --2- 10.20.193.27:0/2232626932 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f186c10ae10 0x7f186c0ef900 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.511 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7f859aa516c0 1 -- 10.20.193.27:0/4073212428 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8594102b90 con 0x7f8594110050 2026-02-13T02:20:26.512 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7f8593fff6c0 1 --2- 10.20.193.27:0/4073212428 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f856803f940 0x7f856805fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.513 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7f186ad766c0 1 --2- 10.20.193.27:0/2232626932 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f186c14d2c0 0x7f186c0f04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.513 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7f186ad766c0 1 -- 10.20.193.27:0/2232626932 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f186c14d2c0 msgr2=0x7f186c0f04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 11 2026-02-13T02:20:26.513 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7f186ad766c0 1 -- 10.20.193.27:0/2232626932 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f186c14d2c0 msgr2=0x7f186c0f04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-13T02:20:26.513 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7f186ad766c0 1 --2- 10.20.193.27:0/2232626932 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f186c14d2c0 0x7f186c0f04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-13T02:20:26.513 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7f186ad766c0 1 --2- 10.20.193.27:0/2232626932 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f186c14d2c0 0x7f186c0f04f0 unknown :-1 s=START_CONNECT pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-13T02:20:26.514 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7f1869d746c0 1 -- 10.20.193.27:0/2232626932 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f186c10ae10 msgr2=0x7f186c0ef900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 10 2026-02-13T02:20:26.514 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7f1869d746c0 1 -- 10.20.193.27:0/2232626932 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f186c10ae10 msgr2=0x7f186c0ef900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-13T02:20:26.514 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7f1869d746c0 1 --2- 10.20.193.27:0/2232626932 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f186c10ae10 0x7f186c0ef900 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-13T02:20:26.514 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7f1869d746c0 1 --2- 10.20.193.27:0/2232626932 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f186c10ae10 0x7f186c0ef900 unknown :-1 s=START_CONNECT pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-13T02:20:26.514 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7f186a5756c0 1 -- 10.20.193.27:0/2232626932 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f186c10ae10 msgr2=0x7f186c0ef900 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-13T02:20:26.514 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7f186a5756c0 1 --2- 10.20.193.27:0/2232626932 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f186c10ae10 0x7f186c0ef900 unknown :-1 s=START_CONNECT pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.514 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7f8593fff6c0 1 --2- 10.20.193.27:0/4073212428 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f856803f940 0x7f856805fdf0 secure :-1 s=READY pgs=58 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8580005fd0 tx=0x7f8580005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.515 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7f186a5756c0 1 -- 10.20.193.27:0/2232626932 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f186c14d2c0 msgr2=0x7f186c0f04f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-13T02:20:26.515 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7f186a5756c0 1 --2- 10.20.193.27:0/2232626932 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f186c14d2c0 0x7f186c0f04f0 unknown :-1 s=START_CONNECT pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.515 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7f186a5756c0 1 -- 10.20.193.27:0/2232626932 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185400a6e0 con 0x7f186c0554b0 2026-02-13T02:20:26.515 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7f186a5756c0 1 --2- 10.20.193.27:0/2232626932 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f186c0554b0 0x7f186c0ef2b0 secure :-1 s=READY pgs=111 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f186000b6c0 tx=0x7f186000bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.516 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7fd506ffd6c0 1 -- 10.20.193.27:0/2920323902 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd50803f8a0 msgr2=0x7fd50805fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.516 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7fd506ffd6c0 1 --2- 10.20.193.27:0/2920323902 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd50803f8a0 0x7fd50805fd50 secure :-1 s=READY pgs=55 gs=6 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd520007240 tx=0x7fd520007620 comp rx=0 tx=0).stop 2026-02-13T02:20:26.516 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7fd506ffd6c0 1 -- 10.20.193.27:0/2920323902 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd534110a00 msgr2=0x7fd5340f8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.517 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7fd506ffd6c0 1 --2- 10.20.193.27:0/2920323902 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd534110a00 0x7fd5340f8db0 secure :-1 s=READY pgs=102 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd528006fd0 tx=0x7fd528003a40 comp rx=0 tx=0).stop 2026-02-13T02:20:26.517 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7fd506ffd6c0 1 -- 10.20.193.27:0/2920323902 shutdown_connections 2026-02-13T02:20:26.517 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7fd506ffd6c0 1 --2- 10.20.193.27:0/2920323902 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd50803f8a0 0x7fd50805fd50 unknown :-1 s=CLOSED pgs=55 gs=6 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.517 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7fd506ffd6c0 1 --2- 10.20.193.27:0/2920323902 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd534110a00 0x7fd5340f8db0 unknown :-1 s=CLOSED pgs=102 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.517 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7fd506ffd6c0 1 --2- 10.20.193.27:0/2920323902 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd534110050 0x7fd534171390 unknown :-1 s=CLOSED pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.517 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7fd506ffd6c0 1 --2- 10.20.193.27:0/2920323902 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd53410eed0 0x7fd5340f8550 unknown :-1 s=CLOSED pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.517 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.496+0000 7fd506ffd6c0 1 -- 10.20.193.27:0/2920323902 >> 10.20.193.27:0/2920323902 conn(0x7fd5340ec5f0 msgr2=0x7fd5340ee640 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.517 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.497+0000 7fd506ffd6c0 1 -- 10.20.193.27:0/2920323902 shutdown_connections 2026-02-13T02:20:26.517 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.497+0000 7fd506ffd6c0 1 -- 10.20.193.27:0/2920323902 wait complete. 2026-02-13T02:20:26.518 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.497+0000 7f18537fe6c0 1 -- 10.20.193.27:0/2232626932 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f18600044a0 con 0x7f186c0554b0 2026-02-13T02:20:26.518 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.497+0000 7f85917fa6c0 1 -- 10.20.193.27:0/4073212428 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 935+0+0 (secure 0 0 0) 0x7f8584038000 con 0x7f8594110050 2026-02-13T02:20:26.519 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.497+0000 7f1870a986c0 1 -- 10.20.193.27:0/2232626932 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f186c0f1290 con 0x7f186c0554b0 2026-02-13T02:20:26.519 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.497+0000 7f1870a986c0 1 -- 10.20.193.27:0/2232626932 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f186c0f17c0 con 0x7f186c0554b0 2026-02-13T02:20:26.519 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.498+0000 7f1870a986c0 1 -- 10.20.193.27:0/2232626932 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f186c057030 con 0x7f186c0554b0 2026-02-13T02:20:26.519 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.498+0000 7f48c9ffb6c0 1 -- 10.20.193.27:0/2718718715 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f48bc0088d0 con 0x7f48d4110050 2026-02-13T02:20:26.519 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.499+0000 7f18537fe6c0 1 -- 10.20.193.27:0/2232626932 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f186000be50 con 0x7f186c0554b0 2026-02-13T02:20:26.520 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.499+0000 7f18537fe6c0 1 -- 10.20.193.27:0/2232626932 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f186000cbf0 con 0x7f186c0554b0 2026-02-13T02:20:26.520 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.499+0000 7f85917fa6c0 1 -- 10.20.193.27:0/4073212428 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f858400d240 con 0x7f8594110050 2026-02-13T02:20:26.520 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.500+0000 7f18537fe6c0 1 -- 10.20.193.27:0/2232626932 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 8) ==== 52814+0+0 (secure 0 0 0) 0x7f1860012080 con 0x7f186c0554b0 2026-02-13T02:20:26.520 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.499+0000 7fb5f871c6c0 1 -- 10.20.193.27:0/2928268519 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb5f010eed0 msgr2=0x7fb5f010f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.521 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.499+0000 7fb5f871c6c0 1 --2- 10.20.193.27:0/2928268519 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb5f010eed0 0x7fb5f010f2c0 secure :-1 s=READY pgs=110 gs=3 cs=0 l=1 c_cookie=8204686331120479973 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb5e0009970 tx=0x7fb5e002ed20 comp rx=0 tx=0).stop 2026-02-13T02:20:26.521 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.500+0000 7f18537fe6c0 1 --2- 10.20.193.27:0/2232626932 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f184403f8f0 0x7f184405fda0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.521 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.500+0000 7f18537fe6c0 1 -- 10.20.193.27:0/2232626932 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 935+0+0 (secure 0 0 0) 0x7f1860052230 con 0x7f186c0554b0 2026-02-13T02:20:26.521 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.500+0000 7f1869d746c0 1 --2- 10.20.193.27:0/2232626932 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f184403f8f0 0x7f184405fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=6 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.522 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.500+0000 7fb5f871c6c0 1 -- 10.20.193.27:0/2928268519 shutdown_connections 2026-02-13T02:20:26.522 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.500+0000 7fb5f871c6c0 1 --2- 10.20.193.27:0/2928268519 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb5f0110a00 0x7fb5f0171390 unknown :-1 s=CLOSED pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.522 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.500+0000 7fb5f871c6c0 1 --2- 10.20.193.27:0/2928268519 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb5f0110050 0x7fb5f01104c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.522 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.500+0000 7fb5f871c6c0 1 --2- 10.20.193.27:0/2928268519 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb5f010eed0 0x7fb5f010f2c0 unknown :-1 s=CLOSED pgs=110 gs=3 cs=0 l=1 c_cookie=8204686331120479973 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.522 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.500+0000 7fb5f871c6c0 1 -- 10.20.193.27:0/2928268519 >> 10.20.193.27:0/2928268519 conn(0x7fb5f00ec5f0 msgr2=0x7fb5f010ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.523 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.500+0000 7f1869d746c0 1 --2- 10.20.193.27:0/2232626932 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f184403f8f0 0x7f184405fda0 secure :-1 s=READY pgs=59 gs=6 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f185400aa10 tx=0x7f18540023a0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.523 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.501+0000 7fb5f871c6c0 1 -- 10.20.193.27:0/2928268519 shutdown_connections 2026-02-13T02:20:26.523 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.503+0000 7f18537fe6c0 1 -- 10.20.193.27:0/2232626932 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f18600128b0 con 0x7f186c0554b0 2026-02-13T02:20:26.524 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.503+0000 7fb5f871c6c0 1 -- 10.20.193.27:0/2928268519 wait complete. 2026-02-13T02:20:26.524 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.503+0000 7fb5f871c6c0 1 Processor -- start 2026-02-13T02:20:26.524 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.503+0000 7fb5f871c6c0 1 -- start start 2026-02-13T02:20:26.524 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.503+0000 7fb5f871c6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb5f010eed0 0x7fb5f00edea0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.524 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.503+0000 7fb5f871c6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb5f0110050 0x7fb5f00ee4f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.524 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.503+0000 7fb5f871c6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb5f0110a00 0x7fb5f00ef0e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.524 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.503+0000 7fb5f871c6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f00ef870 con 0x7fb5f010eed0 2026-02-13T02:20:26.525 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.503+0000 7fb5f871c6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f00ef9e0 con 0x7fb5f0110050 2026-02-13T02:20:26.525 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.503+0000 7fb5f871c6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f00efba0 con 0x7fb5f0110a00 2026-02-13T02:20:26.525 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.503+0000 7fb5f648d6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb5f010eed0 0x7fb5f00edea0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.525 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.503+0000 7fb5f648d6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb5f010eed0 0x7fb5f00edea0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:43752/0 (socket says 10.20.193.27:43752) 2026-02-13T02:20:26.525 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.503+0000 7fb5f6c8e6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb5f0110a00 0x7fb5f00ef0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.525 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.503+0000 7fb5f648d6c0 1 -- 10.20.193.27:0/416812117 learned_addr learned my addr 10.20.193.27:0/416812117 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:20:26.525 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.504+0000 7fb5f6c8e6c0 1 -- 10.20.193.27:0/416812117 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb5f0110a00 msgr2=0x7fb5f00ef0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 10 2026-02-13T02:20:26.525 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.504+0000 7fb5f6c8e6c0 1 -- 10.20.193.27:0/416812117 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb5f0110a00 msgr2=0x7fb5f00ef0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-13T02:20:26.525 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.504+0000 7fb5f6c8e6c0 1 --2- 10.20.193.27:0/416812117 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb5f0110a00 0x7fb5f00ef0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-13T02:20:26.525 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.504+0000 7fb5f6c8e6c0 1 --2- 10.20.193.27:0/416812117 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb5f0110a00 0x7fb5f00ef0e0 unknown :-1 s=START_CONNECT pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-13T02:20:26.525 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.504+0000 7fb5f648d6c0 1 -- 10.20.193.27:0/416812117 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb5f0110a00 msgr2=0x7fb5f00ef0e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-13T02:20:26.525 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.504+0000 7fb5f648d6c0 1 --2- 10.20.193.27:0/416812117 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb5f0110a00 0x7fb5f00ef0e0 unknown :-1 s=START_CONNECT pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.525 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.504+0000 7fb5f648d6c0 1 -- 10.20.193.27:0/416812117 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb5f0110050 msgr2=0x7fb5f00ee4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.526 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.504+0000 7fb5f648d6c0 1 --2- 10.20.193.27:0/416812117 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb5f0110050 0x7fb5f00ee4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.526 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.504+0000 7fb5f648d6c0 1 -- 10.20.193.27:0/416812117 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e0009590 con 0x7fb5f010eed0 2026-02-13T02:20:26.526 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.504+0000 7fb5f648d6c0 1 --2- 10.20.193.27:0/416812117 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb5f010eed0 0x7fb5f00edea0 secure :-1 s=READY pgs=112 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb5e00098c0 tx=0x7fb5e0054a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.526 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.504+0000 7fb5d77fe6c0 1 -- 10.20.193.27:0/416812117 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb5e0054ea0 con 0x7fb5f010eed0 2026-02-13T02:20:26.526 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.504+0000 7fb5d77fe6c0 1 -- 10.20.193.27:0/416812117 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb5e002f6c0 con 0x7fb5f010eed0 2026-02-13T02:20:26.526 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.504+0000 7fb5f871c6c0 1 -- 10.20.193.27:0/416812117 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f00efe20 con 0x7fb5f010eed0 2026-02-13T02:20:26.526 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.504+0000 7fb5d77fe6c0 1 -- 10.20.193.27:0/416812117 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb5e0002a30 con 0x7fb5f010eed0 2026-02-13T02:20:26.526 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.504+0000 7fb5f871c6c0 1 -- 10.20.193.27:0/416812117 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f00f0350 con 0x7fb5f010eed0 2026-02-13T02:20:26.526 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.505+0000 7fb5f871c6c0 1 -- 10.20.193.27:0/416812117 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5f01752c0 con 0x7fb5f010eed0 2026-02-13T02:20:26.526 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.505+0000 7fb5d77fe6c0 1 -- 10.20.193.27:0/416812117 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 8) ==== 52814+0+0 (secure 0 0 0) 0x7fb5e0002bc0 con 0x7fb5f010eed0 2026-02-13T02:20:26.526 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.505+0000 7fb5d77fe6c0 1 --2- 10.20.193.27:0/416812117 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb5cc03f8f0 0x7fb5cc05fda0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.526 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.505+0000 7fb5d77fe6c0 1 -- 10.20.193.27:0/416812117 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 935+0+0 (secure 0 0 0) 0x7fb5e0098030 con 0x7fb5f010eed0 2026-02-13T02:20:26.526 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.506+0000 7fb5f5c8c6c0 1 --2- 10.20.193.27:0/416812117 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb5cc03f8f0 0x7fb5cc05fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.526 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.506+0000 7fb5f5c8c6c0 1 --2- 10.20.193.27:0/416812117 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb5cc03f8f0 0x7fb5cc05fda0 secure :-1 s=READY pgs=60 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb5e4005980 tx=0x7fb5e4005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.527 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:26.527 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.508+0000 7fb5d77fe6c0 1 -- 10.20.193.27:0/416812117 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fb5e00071c0 con 0x7fb5f010eed0 2026-02-13T02:20:26.543 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '707dbb29-0882-11f1-9262-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='trial027') 2026-02-13T02:20:26.543 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '707dbb29-0882-11f1-9262-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='trial027') 2026-02-13T02:20:26.561 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '707dbb29-0882-11f1-9262-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='trial027') 2026-02-13T02:20:26.567 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.565+0000 7fb8ebf536c0 1 -- 10.20.193.27:0/2673424342 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7fb8e41104c0 con 0x7fb8e410eed0 2026-02-13T02:20:26.571 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.569+0000 7fb8d2ffd6c0 1 -- 10.20.193.27:0/2673424342 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v19)=0 v19) ==== 116+0+0 (secure 0 0 0) 0x7fb8d40123b0 con 0x7fb8e410eed0 2026-02-13T02:20:26.574 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.573+0000 7fb8ebf536c0 1 -- 10.20.193.27:0/2673424342 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb8c003f8f0 msgr2=0x7fb8c005fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.574 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.573+0000 7fb8ebf536c0 1 --2- 10.20.193.27:0/2673424342 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb8c003f8f0 0x7fb8c005fda0 secure :-1 s=READY pgs=56 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb8d8009260 tx=0x7fb8d8009050 comp rx=0 tx=0).stop 2026-02-13T02:20:26.574 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.573+0000 7fb8ebf536c0 1 -- 10.20.193.27:0/2673424342 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb8e410eed0 msgr2=0x7fb8e40f8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.575 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.573+0000 7fb8ebf536c0 1 --2- 10.20.193.27:0/2673424342 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb8e410eed0 0x7fb8e40f8580 secure :-1 s=READY pgs=104 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb8d400b440 tx=0x7fb8d400b910 comp rx=0 tx=0).stop 2026-02-13T02:20:26.575 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.574+0000 7fb8ebf536c0 1 -- 10.20.193.27:0/2673424342 shutdown_connections 2026-02-13T02:20:26.576 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.574+0000 7fb8ebf536c0 1 --2- 10.20.193.27:0/2673424342 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb8c003f8f0 0x7fb8c005fda0 unknown :-1 s=CLOSED pgs=56 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.576 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.574+0000 7fb8ebf536c0 1 --2- 10.20.193.27:0/2673424342 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb8e4110a00 0x7fb8e40f8de0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.576 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.574+0000 7fb8ebf536c0 1 --2- 10.20.193.27:0/2673424342 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb8e4110050 0x7fb8e4171390 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.576 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.574+0000 7fb8ebf536c0 1 --2- 10.20.193.27:0/2673424342 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb8e410eed0 0x7fb8e40f8580 unknown :-1 s=CLOSED pgs=104 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.576 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.574+0000 7fb8ebf536c0 1 -- 10.20.193.27:0/2673424342 >> 10.20.193.27:0/2673424342 conn(0x7fb8e40ec5f0 msgr2=0x7fb8e40eeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.581 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.580+0000 7fb8ebf536c0 1 -- 10.20.193.27:0/2673424342 shutdown_connections 2026-02-13T02:20:26.585 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.584+0000 7fb8ebf536c0 1 -- 10.20.193.27:0/2673424342 wait complete. 2026-02-13T02:20:26.593 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.588+0000 7fe99b32f6c0 1 -- 10.20.193.27:0/4235678119 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe99410ddd0 msgr2=0x7fe99410e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.593 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.588+0000 7fe99b32f6c0 1 --2- 10.20.193.27:0/4235678119 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe99410ddd0 0x7fe99410e240 secure :-1 s=READY pgs=113 gs=1 cs=0 l=1 c_cookie=17583656383044717696 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe988009970 tx=0x7fe98802ed20 comp rx=0 tx=0).stop 2026-02-13T02:20:26.594 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.589+0000 7fe99b32f6c0 1 -- 10.20.193.27:0/4235678119 shutdown_connections 2026-02-13T02:20:26.594 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.589+0000 7fe99b32f6c0 1 --2- 10.20.193.27:0/4235678119 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe99410e780 0x7fe99416f110 unknown :-1 s=CLOSED pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.594 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.589+0000 7fe99b32f6c0 1 --2- 10.20.193.27:0/4235678119 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe99410ddd0 0x7fe99410e240 unknown :-1 s=CLOSED pgs=113 gs=1 cs=0 l=1 c_cookie=17583656383044717696 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.594 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.589+0000 7fe99b32f6c0 1 --2- 10.20.193.27:0/4235678119 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe99410cc50 0x7fe99410d040 unknown :-1 s=CLOSED pgs=0 gs=5 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.594 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.589+0000 7fe99b32f6c0 1 -- 10.20.193.27:0/4235678119 >> 10.20.193.27:0/4235678119 conn(0x7fe9940ea370 msgr2=0x7fe99410a780 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.594 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.589+0000 7fe99b32f6c0 1 -- 10.20.193.27:0/4235678119 shutdown_connections 2026-02-13T02:20:26.594 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.589+0000 7fe99b32f6c0 1 -- 10.20.193.27:0/4235678119 wait complete. 2026-02-13T02:20:26.594 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.589+0000 7fe99b32f6c0 1 Processor -- start 2026-02-13T02:20:26.594 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.589+0000 7fe99b32f6c0 1 -- start start 2026-02-13T02:20:26.594 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.589+0000 7fe99b32f6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe99410cc50 0x7fe994061ea0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.594 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.589+0000 7fe99b32f6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe99410ddd0 0x7fe99416f0f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.594 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.589+0000 7fe99b32f6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe99410e780 0x7fe9940607a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.595 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.589+0000 7fe99b32f6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe994060da0 con 0x7fe99410cc50 2026-02-13T02:20:26.595 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.589+0000 7fe99b32f6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe994060f10 con 0x7fe99410ddd0 2026-02-13T02:20:26.595 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.589+0000 7fe99b32f6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe9940610d0 con 0x7fe99410e780 2026-02-13T02:20:26.595 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.590+0000 7fe9990a06c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe99410cc50 0x7fe994061ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.595 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.590+0000 7fe9998a16c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe99410e780 0x7fe9940607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.595 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.590+0000 7fe99889f6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe99410ddd0 0x7fe99416f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.595 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.590+0000 7fe9990a06c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe99410cc50 0x7fe994061ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:43780/0 (socket says 10.20.193.27:43780) 2026-02-13T02:20:26.595 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.590+0000 7fe9990a06c0 1 -- 10.20.193.27:0/1094763007 learned_addr learned my addr 10.20.193.27:0/1094763007 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:20:26.595 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.590+0000 7fe99889f6c0 1 -- 10.20.193.27:0/1094763007 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe99410ddd0 msgr2=0x7fe99416f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 10 2026-02-13T02:20:26.595 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.590+0000 7fe99889f6c0 1 -- 10.20.193.27:0/1094763007 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe99410ddd0 msgr2=0x7fe99416f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-13T02:20:26.595 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.590+0000 7fe99889f6c0 1 --2- 10.20.193.27:0/1094763007 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe99410ddd0 0x7fe99416f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-13T02:20:26.595 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.590+0000 7fe9998a16c0 1 -- 10.20.193.27:0/1094763007 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe99410e780 msgr2=0x7fe9940607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 11 2026-02-13T02:20:26.596 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.590+0000 7fe9998a16c0 1 -- 10.20.193.27:0/1094763007 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe99410e780 msgr2=0x7fe9940607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-13T02:20:26.596 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.590+0000 7fe9998a16c0 1 --2- 10.20.193.27:0/1094763007 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe99410e780 0x7fe9940607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-13T02:20:26.596 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.590+0000 7fe9998a16c0 1 --2- 10.20.193.27:0/1094763007 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe99410e780 0x7fe9940607a0 unknown :-1 s=START_CONNECT pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-13T02:20:26.596 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.590+0000 7fe9990a06c0 1 -- 10.20.193.27:0/1094763007 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe99410e780 msgr2=0x7fe9940607a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-13T02:20:26.596 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.590+0000 7fe9990a06c0 1 --2- 10.20.193.27:0/1094763007 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe99410e780 0x7fe9940607a0 unknown :-1 s=START_CONNECT pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.596 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.590+0000 7fe9990a06c0 1 -- 10.20.193.27:0/1094763007 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe99410ddd0 msgr2=0x7fe99416f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.596 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.590+0000 7fe99889f6c0 1 --2- 10.20.193.27:0/1094763007 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe99410ddd0 0x7fe99416f0f0 unknown :-1 s=START_CONNECT pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-13T02:20:26.596 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.590+0000 7fe9990a06c0 1 --2- 10.20.193.27:0/1094763007 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe99410ddd0 0x7fe99416f0f0 unknown :-1 s=START_CONNECT pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.596 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.590+0000 7fe9990a06c0 1 -- 10.20.193.27:0/1094763007 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe988009590 con 0x7fe99410cc50 2026-02-13T02:20:26.596 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.590+0000 7fe9990a06c0 1 --2- 10.20.193.27:0/1094763007 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe99410cc50 0x7fe994061ea0 secure :-1 s=READY pgs=114 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe984005dc0 tx=0x7fe98400bbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.596 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.590+0000 7fe9827fc6c0 1 -- 10.20.193.27:0/1094763007 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe984004250 con 0x7fe99410cc50 2026-02-13T02:20:26.597 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.590+0000 7fe99b32f6c0 1 -- 10.20.193.27:0/1094763007 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9940613b0 con 0x7fe99410cc50 2026-02-13T02:20:26.597 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.590+0000 7fe9827fc6c0 1 -- 10.20.193.27:0/1094763007 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fe98400be30 con 0x7fe99410cc50 2026-02-13T02:20:26.597 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.590+0000 7fe9827fc6c0 1 -- 10.20.193.27:0/1094763007 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe984004250 con 0x7fe99410cc50 2026-02-13T02:20:26.597 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.590+0000 7fe99b32f6c0 1 -- 10.20.193.27:0/1094763007 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9940fa5b0 con 0x7fe99410cc50 2026-02-13T02:20:26.597 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.591+0000 7fe9827fc6c0 1 -- 10.20.193.27:0/1094763007 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 8) ==== 52814+0+0 (secure 0 0 0) 0x7fe9840028c0 con 0x7fe99410cc50 2026-02-13T02:20:26.597 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.591+0000 7fe9827fc6c0 1 --2- 10.20.193.27:0/1094763007 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe97003f950 0x7fe97005fe00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.597 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.591+0000 7fe9827fc6c0 1 -- 10.20.193.27:0/1094763007 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 935+0+0 (secure 0 0 0) 0x7fe984012240 con 0x7fe99410cc50 2026-02-13T02:20:26.597 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.592+0000 7fe99889f6c0 1 --2- 10.20.193.27:0/1094763007 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe97003f950 0x7fe97005fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=6 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.592+0000 7fe99889f6c0 1 --2- 10.20.193.27:0/1094763007 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe97003f950 0x7fe97005fe00 secure :-1 s=READY pgs=61 gs=6 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe9880023e0 tx=0x7fe98802ff90 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.592+0000 7fe99b32f6c0 1 -- 10.20.193.27:0/1094763007 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9941730b0 con 0x7fe99410cc50 2026-02-13T02:20:26.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.594+0000 7fe9827fc6c0 1 -- 10.20.193.27:0/1094763007 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fe98400d7d0 con 0x7fe99410cc50 2026-02-13T02:20:26.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.595+0000 7fe0d8d196c0 1 -- 10.20.193.27:0/3880409310 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe0d410ddd0 msgr2=0x7fe0d410e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.596+0000 7fe0d15736c0 1 -- 10.20.193.27:0/3880409310 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe0bc004560 con 0x7fe0d410ddd0 2026-02-13T02:20:26.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.596+0000 7fe0d8d196c0 1 --2- 10.20.193.27:0/3880409310 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe0d410ddd0 0x7fe0d410e240 secure :-1 s=READY pgs=115 gs=1 cs=0 l=1 c_cookie=5368074816216145196 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe0bc009970 tx=0x7fe0bc02ed50 comp rx=0 tx=0).stop 2026-02-13T02:20:26.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.596+0000 7fe0d8d196c0 1 -- 10.20.193.27:0/3880409310 shutdown_connections 2026-02-13T02:20:26.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.596+0000 7fe0d8d196c0 1 --2- 10.20.193.27:0/3880409310 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe0d410e780 0x7fe0d416f110 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.596+0000 7fe0d8d196c0 1 --2- 10.20.193.27:0/3880409310 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe0d410ddd0 0x7fe0d410e240 unknown :-1 s=CLOSED pgs=115 gs=1 cs=0 l=1 c_cookie=5368074816216145196 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.596+0000 7fe0d8d196c0 1 --2- 10.20.193.27:0/3880409310 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe0d410cc50 0x7fe0d410d040 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.596+0000 7fe0d8d196c0 1 -- 10.20.193.27:0/3880409310 >> 10.20.193.27:0/3880409310 conn(0x7fe0d40ea370 msgr2=0x7fe0d410a780 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.596+0000 7fe0d8d196c0 1 -- 10.20.193.27:0/3880409310 shutdown_connections 2026-02-13T02:20:26.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.596+0000 7fe0d8d196c0 1 -- 10.20.193.27:0/3880409310 wait complete. 2026-02-13T02:20:26.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.596+0000 7fe0d8d196c0 1 Processor -- start 2026-02-13T02:20:26.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.597+0000 7fe0d8d196c0 1 -- start start 2026-02-13T02:20:26.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.597+0000 7fe0d8d196c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe0d410cc50 0x7fe0d40fbd40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.597+0000 7fe0d8d196c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe0d410ddd0 0x7fe0d416f110 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.597+0000 7fe0d25756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe0d410cc50 0x7fe0d40fbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.597+0000 7fe0d8d196c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe0d410e780 0x7fe0d40fc4f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.597+0000 7fe0d25756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe0d410cc50 0x7fe0d40fbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:43800/0 (socket says 10.20.193.27:43800) 2026-02-13T02:20:26.603 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.597+0000 7fe0d25756c0 1 -- 10.20.193.27:0/1855195464 learned_addr learned my addr 10.20.193.27:0/1855195464 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:20:26.603 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.597+0000 7fe0d8d196c0 1 -- 10.20.193.27:0/1855195464 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d40fcc80 con 0x7fe0d410cc50 2026-02-13T02:20:26.603 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.597+0000 7fe0d8d196c0 1 -- 10.20.193.27:0/1855195464 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d40fcdf0 con 0x7fe0d410e780 2026-02-13T02:20:26.603 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.597+0000 7fe0d8d196c0 1 -- 10.20.193.27:0/1855195464 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d40fcfb0 con 0x7fe0d410ddd0 2026-02-13T02:20:26.603 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.597+0000 7fe0d2d766c0 1 --2- 10.20.193.27:0/1855195464 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe0d410e780 0x7fe0d40fc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.603 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.597+0000 7fe0d25756c0 1 -- 10.20.193.27:0/1855195464 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe0d410ddd0 msgr2=0x7fe0d416f110 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2026-02-13T02:20:26.603 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.597+0000 7fe0d25756c0 1 --2- 10.20.193.27:0/1855195464 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe0d410ddd0 0x7fe0d416f110 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.603 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.597+0000 7fe0d25756c0 1 -- 10.20.193.27:0/1855195464 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe0d410e780 msgr2=0x7fe0d40fc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.603 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.597+0000 7fe0d25756c0 1 --2- 10.20.193.27:0/1855195464 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe0d410e780 0x7fe0d40fc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.603 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.597+0000 7fe0d25756c0 1 -- 10.20.193.27:0/1855195464 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0bc009590 con 0x7fe0d410cc50 2026-02-13T02:20:26.603 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.597+0000 7fe0d25756c0 1 --2- 10.20.193.27:0/1855195464 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe0d410cc50 0x7fe0d40fbd40 secure :-1 s=READY pgs=116 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe0c800b750 tx=0x7fe0c800bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.603 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.598+0000 7fe0bb7fe6c0 1 -- 10.20.193.27:0/1855195464 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe0c8004290 con 0x7fe0d410cc50 2026-02-13T02:20:26.604 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.598+0000 7fe0d8d196c0 1 -- 10.20.193.27:0/1855195464 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0d40fd290 con 0x7fe0d410cc50 2026-02-13T02:20:26.604 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.598+0000 7fe0bb7fe6c0 1 -- 10.20.193.27:0/1855195464 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fe0c80043f0 con 0x7fe0d410cc50 2026-02-13T02:20:26.604 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.598+0000 7fe0d8d196c0 1 -- 10.20.193.27:0/1855195464 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0d40fd7c0 con 0x7fe0d410cc50 2026-02-13T02:20:26.604 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.598+0000 7fe0bb7fe6c0 1 -- 10.20.193.27:0/1855195464 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe0c800cd10 con 0x7fe0d410cc50 2026-02-13T02:20:26.604 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.599+0000 7f3ef2d5c6c0 1 -- 10.20.193.27:0/3079469529 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3eec110050 msgr2=0x7f3eec1104c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.604 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.599+0000 7f3ef2d5c6c0 1 --2- 10.20.193.27:0/3079469529 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3eec110050 0x7f3eec1104c0 secure :-1 s=READY pgs=117 gs=2 cs=0 l=1 c_cookie=5508541918905289608 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3ed4009970 tx=0x7f3ed402ed20 comp rx=0 tx=0).stop 2026-02-13T02:20:26.605 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.599+0000 7fe0bb7fe6c0 1 -- 10.20.193.27:0/1855195464 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 8) ==== 52814+0+0 (secure 0 0 0) 0x7fe0c8013070 con 0x7fe0d410cc50 2026-02-13T02:20:26.605 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.599+0000 7fe0bb7fe6c0 1 --2- 10.20.193.27:0/1855195464 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe0ac03f870 0x7fe0ac05fd20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.605 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.599+0000 7fe0d1d746c0 1 --2- 10.20.193.27:0/1855195464 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe0ac03f870 0x7fe0ac05fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.606 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.599+0000 7f3ef2d5c6c0 1 -- 10.20.193.27:0/3079469529 shutdown_connections 2026-02-13T02:20:26.606 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.599+0000 7f3ef2d5c6c0 1 --2- 10.20.193.27:0/3079469529 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3eec110a00 0x7f3eec171390 unknown :-1 s=CLOSED pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.606 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.599+0000 7f3ef2d5c6c0 1 --2- 10.20.193.27:0/3079469529 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3eec110050 0x7f3eec1104c0 unknown :-1 s=CLOSED pgs=117 gs=2 cs=0 l=1 c_cookie=5508541918905289608 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.606 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.599+0000 7f3ef2d5c6c0 1 --2- 10.20.193.27:0/3079469529 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3eec10eed0 0x7f3eec10f2c0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.606 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.599+0000 7f3ef2d5c6c0 1 -- 10.20.193.27:0/3079469529 >> 10.20.193.27:0/3079469529 conn(0x7f3eec0ec5f0 msgr2=0x7f3eec10ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.606 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.599+0000 7f3ef2d5c6c0 1 -- 10.20.193.27:0/3079469529 shutdown_connections 2026-02-13T02:20:26.606 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.599+0000 7f3ef2d5c6c0 1 -- 10.20.193.27:0/3079469529 wait complete. 2026-02-13T02:20:26.607 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.599+0000 7fe0d8d196c0 1 -- 10.20.193.27:0/1855195464 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0d4170e80 con 0x7fe0d410cc50 2026-02-13T02:20:26.607 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.600+0000 7fe0d1d746c0 1 --2- 10.20.193.27:0/1855195464 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe0ac03f870 0x7fe0ac05fd20 secure :-1 s=READY pgs=62 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe0bc0098c0 tx=0x7fe0bc005ae0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.607 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.600+0000 7fe0bb7fe6c0 1 -- 10.20.193.27:0/1855195464 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 935+0+0 (secure 0 0 0) 0x7fe0c8054440 con 0x7fe0d410cc50 2026-02-13T02:20:26.609 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.600+0000 7f3ef2d5c6c0 1 Processor -- start 2026-02-13T02:20:26.609 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.600+0000 7f3ef2d5c6c0 1 -- start start 2026-02-13T02:20:26.609 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.600+0000 7f3ef2d5c6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3eec10eed0 0x7f3eec0fc8f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.609 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.600+0000 7f3ef2d5c6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3eec110050 0x7f3eec171390 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.609 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.600+0000 7f3ef2d5c6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3eec110a00 0x7f3eec0fd150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.609 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.600+0000 7f3ef2d5c6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec0fdd80 con 0x7f3eec10eed0 2026-02-13T02:20:26.609 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.600+0000 7f3ef2d5c6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec0fdef0 con 0x7f3eec110050 2026-02-13T02:20:26.609 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.600+0000 7f3ef2d5c6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec0fe0b0 con 0x7f3eec110a00 2026-02-13T02:20:26.609 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.600+0000 7f3ef0ad16c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3eec10eed0 0x7f3eec0fc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.600+0000 7f3ef0ad16c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3eec10eed0 0x7f3eec0fc8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:43810/0 (socket says 10.20.193.27:43810) 2026-02-13T02:20:26.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.600+0000 7f3ef0ad16c0 1 -- 10.20.193.27:0/2445171217 learned_addr learned my addr 10.20.193.27:0/2445171217 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:20:26.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.600+0000 7f3ef0ad16c0 1 -- 10.20.193.27:0/2445171217 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3eec110a00 msgr2=0x7f3eec0fd150 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2026-02-13T02:20:26.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.600+0000 7f3ef0ad16c0 1 --2- 10.20.193.27:0/2445171217 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3eec110a00 0x7f3eec0fd150 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.600+0000 7f3ef0ad16c0 1 -- 10.20.193.27:0/2445171217 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3eec110050 msgr2=0x7f3eec171390 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2026-02-13T02:20:26.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.600+0000 7f3ef0ad16c0 1 --2- 10.20.193.27:0/2445171217 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3eec110050 0x7f3eec171390 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.600+0000 7f3ef0ad16c0 1 -- 10.20.193.27:0/2445171217 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed4009590 con 0x7f3eec10eed0 2026-02-13T02:20:26.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.600+0000 7f3ef0ad16c0 1 --2- 10.20.193.27:0/2445171217 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3eec10eed0 0x7f3eec0fc8f0 secure :-1 s=READY pgs=118 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3edc00b710 tx=0x7f3edc00bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.603+0000 7f3ee1ffb6c0 1 -- 10.20.193.27:0/2445171217 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3edc004250 con 0x7f3eec10eed0 2026-02-13T02:20:26.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.603+0000 7f3ee1ffb6c0 1 -- 10.20.193.27:0/2445171217 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3edc009070 con 0x7f3eec10eed0 2026-02-13T02:20:26.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.603+0000 7f3ee1ffb6c0 1 -- 10.20.193.27:0/2445171217 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3edc00cd40 con 0x7f3eec10eed0 2026-02-13T02:20:26.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.603+0000 7fe0bb7fe6c0 1 -- 10.20.193.27:0/1855195464 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fe0c8003240 con 0x7fe0d410cc50 2026-02-13T02:20:26.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.603+0000 7f5271fab6c0 1 -- 10.20.193.27:0/4245756962 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f526c110050 msgr2=0x7f526c1104c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.603+0000 7f5271fab6c0 1 --2- 10.20.193.27:0/4245756962 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f526c110050 0x7f526c1104c0 secure :-1 s=READY pgs=119 gs=1 cs=0 l=1 c_cookie=11820600559548444642 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f525c009970 tx=0x7f525c02ed20 comp rx=0 tx=0).stop 2026-02-13T02:20:26.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.604+0000 7f5271fab6c0 1 -- 10.20.193.27:0/4245756962 shutdown_connections 2026-02-13T02:20:26.612 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.604+0000 7f5271fab6c0 1 --2- 10.20.193.27:0/4245756962 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f526c110a00 0x7f526c171390 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.612 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.604+0000 7f5271fab6c0 1 --2- 10.20.193.27:0/4245756962 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f526c110050 0x7f526c1104c0 unknown :-1 s=CLOSED pgs=119 gs=1 cs=0 l=1 c_cookie=11820600559548444642 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.612 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.604+0000 7f5271fab6c0 1 --2- 10.20.193.27:0/4245756962 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f526c10eed0 0x7f526c10f2c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.612 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.604+0000 7f5271fab6c0 1 -- 10.20.193.27:0/4245756962 >> 10.20.193.27:0/4245756962 conn(0x7f526c0ec5f0 msgr2=0x7f526c10ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.612 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.604+0000 7f5271fab6c0 1 -- 10.20.193.27:0/4245756962 shutdown_connections 2026-02-13T02:20:26.612 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.604+0000 7f5271fab6c0 1 -- 10.20.193.27:0/4245756962 wait complete. 2026-02-13T02:20:26.612 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.603+0000 7f3ef2d5c6c0 1 -- 10.20.193.27:0/2445171217 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eec0fe390 con 0x7f3eec10eed0 2026-02-13T02:20:26.612 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.603+0000 7f3ef2d5c6c0 1 -- 10.20.193.27:0/2445171217 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eec0fe820 con 0x7f3eec10eed0 2026-02-13T02:20:26.613 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.604+0000 7f3ef2d5c6c0 1 -- 10.20.193.27:0/2445171217 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eec1752c0 con 0x7f3eec10eed0 2026-02-13T02:20:26.613 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.604+0000 7f5271fab6c0 1 Processor -- start 2026-02-13T02:20:26.613 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.604+0000 7f5271fab6c0 1 -- start start 2026-02-13T02:20:26.613 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.604+0000 7f5271fab6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f526c10eed0 0x7f526c0ff050 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.614 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.604+0000 7f5271fab6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f526c110050 0x7f526c171390 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.614 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.604+0000 7f5271fab6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f526c110a00 0x7f526c0ff8b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.614 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.604+0000 7f5271fab6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f526c100270 con 0x7f526c110050 2026-02-13T02:20:26.614 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.604+0000 7f5271fab6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f526c1003b0 con 0x7f526c10eed0 2026-02-13T02:20:26.614 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.604+0000 7f5271fab6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f526c100570 con 0x7f526c110a00 2026-02-13T02:20:26.614 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.605+0000 7f526bfff6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f526c110a00 0x7f526c0ff8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.614 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.605+0000 7f526bfff6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f526c110a00 0x7f526c0ff8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.107:3300/0 says I am v2:10.20.193.27:41712/0 (socket says 10.20.193.27:41712) 2026-02-13T02:20:26.614 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.605+0000 7f526bfff6c0 1 -- 10.20.193.27:0/357666803 learned_addr learned my addr 10.20.193.27:0/357666803 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:20:26.614 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.605+0000 7f526bfff6c0 1 -- 10.20.193.27:0/357666803 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f526c110a00 msgr2=0x7f526c0ff8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 9 2026-02-13T02:20:26.614 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.605+0000 7f526bfff6c0 1 -- 10.20.193.27:0/357666803 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f526c110a00 msgr2=0x7f526c0ff8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-13T02:20:26.614 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.605+0000 7f526bfff6c0 1 --2- 10.20.193.27:0/357666803 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f526c110a00 0x7f526c0ff8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-13T02:20:26.615 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.605+0000 7f526bfff6c0 1 --2- 10.20.193.27:0/357666803 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f526c110a00 0x7f526c0ff8b0 unknown :-1 s=START_CONNECT pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-13T02:20:26.615 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.605+0000 7f3ee1ffb6c0 1 -- 10.20.193.27:0/2445171217 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 8) ==== 52814+0+0 (secure 0 0 0) 0x7f3edc0028c0 con 0x7f3eec10eed0 2026-02-13T02:20:26.615 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.605+0000 7f3ee1ffb6c0 1 --2- 10.20.193.27:0/2445171217 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3ec803f940 0x7f3ec805fdf0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.615 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.605+0000 7f526b7fe6c0 1 --2- 10.20.193.27:0/357666803 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f526c10eed0 0x7f526c0ff050 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.615 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.605+0000 7f526affd6c0 1 --2- 10.20.193.27:0/357666803 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f526c110050 0x7f526c171390 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.616 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.605+0000 7f526b7fe6c0 1 -- 10.20.193.27:0/357666803 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f526c10eed0 msgr2=0x7f526c0ff050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 11 2026-02-13T02:20:26.616 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.605+0000 7f526b7fe6c0 1 -- 10.20.193.27:0/357666803 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f526c10eed0 msgr2=0x7f526c0ff050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-13T02:20:26.616 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.605+0000 7f526b7fe6c0 1 --2- 10.20.193.27:0/357666803 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f526c10eed0 0x7f526c0ff050 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-13T02:20:26.616 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.605+0000 7f3ee1ffb6c0 1 -- 10.20.193.27:0/2445171217 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 935+0+0 (secure 0 0 0) 0x7f3edc051230 con 0x7f3eec10eed0 2026-02-13T02:20:26.616 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.605+0000 7f3ee3fff6c0 1 --2- 10.20.193.27:0/2445171217 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3ec803f940 0x7f3ec805fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.617 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.605+0000 7f526b7fe6c0 1 --2- 10.20.193.27:0/357666803 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f526c10eed0 0x7f526c0ff050 unknown :-1 s=START_CONNECT pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-13T02:20:26.617 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.605+0000 7f526affd6c0 1 -- 10.20.193.27:0/357666803 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f526c110a00 msgr2=0x7f526c0ff8b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-13T02:20:26.617 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.605+0000 7f526affd6c0 1 --2- 10.20.193.27:0/357666803 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f526c110a00 0x7f526c0ff8b0 unknown :-1 s=START_CONNECT pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.617 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.605+0000 7f526affd6c0 1 -- 10.20.193.27:0/357666803 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f526c10eed0 msgr2=0x7f526c0ff050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-13T02:20:26.617 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.605+0000 7f526affd6c0 1 --2- 10.20.193.27:0/357666803 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f526c10eed0 0x7f526c0ff050 unknown :-1 s=START_CONNECT pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.617 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.605+0000 7f526affd6c0 1 -- 10.20.193.27:0/357666803 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f525c009590 con 0x7f526c110050 2026-02-13T02:20:26.617 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.606+0000 7f3ee3fff6c0 1 --2- 10.20.193.27:0/2445171217 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3ec803f940 0x7f3ec805fdf0 secure :-1 s=READY pgs=63 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3ed40098c0 tx=0x7f3ed4005c60 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.618 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.606+0000 7f526affd6c0 1 --2- 10.20.193.27:0/357666803 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f526c110050 0x7f526c171390 secure :-1 s=READY pgs=121 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f525c009970 tx=0x7f525c0028e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.618 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.608+0000 7f5268ff96c0 1 -- 10.20.193.27:0/357666803 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f525c002cf0 con 0x7f526c110050 2026-02-13T02:20:26.618 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.608+0000 7f5268ff96c0 1 -- 10.20.193.27:0/357666803 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f525c007680 con 0x7f526c110050 2026-02-13T02:20:26.618 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.608+0000 7f5268ff96c0 1 -- 10.20.193.27:0/357666803 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f525c037500 con 0x7f526c110050 2026-02-13T02:20:26.618 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.609+0000 7f5271fab6c0 1 -- 10.20.193.27:0/357666803 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f526c1007f0 con 0x7f526c110050 2026-02-13T02:20:26.618 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.609+0000 7f5271fab6c0 1 -- 10.20.193.27:0/357666803 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f526c100c20 con 0x7f526c110050 2026-02-13T02:20:26.618 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.608+0000 7f3ee1ffb6c0 1 -- 10.20.193.27:0/2445171217 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f3edc010e30 con 0x7f3eec10eed0 2026-02-13T02:20:26.619 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.609+0000 7f5271fab6c0 1 -- 10.20.193.27:0/357666803 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f526c100f50 con 0x7f526c110050 2026-02-13T02:20:26.619 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.610+0000 7fad9a87c6c0 1 -- 10.20.193.27:0/2361254777 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fad9410cda0 msgr2=0x7fad9416d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.619 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.610+0000 7fad9a87c6c0 1 --2- 10.20.193.27:0/2361254777 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fad9410cda0 0x7fad9416d730 secure :-1 s=READY pgs=120 gs=2 cs=0 l=1 c_cookie=13177286857054892338 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fad88006fd0 tx=0x7fad8802fd40 comp rx=0 tx=0).stop 2026-02-13T02:20:26.620 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.610+0000 7f5268ff96c0 1 -- 10.20.193.27:0/357666803 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 8) ==== 52814+0+0 (secure 0 0 0) 0x7f525c007cb0 con 0x7f526c110050 2026-02-13T02:20:26.620 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.610+0000 7f5268ff96c0 1 --2- 10.20.193.27:0/357666803 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f524003f8f0 0x7f524005fda0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.620 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.610+0000 7f5268ff96c0 1 -- 10.20.193.27:0/357666803 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 935+0+0 (secure 0 0 0) 0x7f525c075650 con 0x7f526c110050 2026-02-13T02:20:26.620 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.610+0000 7f526b7fe6c0 1 --2- 10.20.193.27:0/357666803 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f524003f8f0 0x7f524005fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=6 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.620 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.611+0000 7f526b7fe6c0 1 --2- 10.20.193.27:0/357666803 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f524003f8f0 0x7f524005fda0 secure :-1 s=READY pgs=64 gs=6 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5258006b60 tx=0x7f5258006af0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.621 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.611+0000 7fad9a87c6c0 1 -- 10.20.193.27:0/2361254777 shutdown_connections 2026-02-13T02:20:26.621 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.611+0000 7fad9a87c6c0 1 --2- 10.20.193.27:0/2361254777 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fad9410cda0 0x7fad9416d730 unknown :-1 s=CLOSED pgs=120 gs=2 cs=0 l=1 c_cookie=13177286857054892338 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.621 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.611+0000 7fad9a87c6c0 1 --2- 10.20.193.27:0/2361254777 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fad9410c3f0 0x7fad9410c860 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.621 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.611+0000 7fad9a87c6c0 1 --2- 10.20.193.27:0/2361254777 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fad9410b270 0x7fad9410b660 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.621 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.611+0000 7fad9a87c6c0 1 -- 10.20.193.27:0/2361254777 >> 10.20.193.27:0/2361254777 conn(0x7fad940e8990 msgr2=0x7fad94108da0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.621 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.611+0000 7fad9a87c6c0 1 -- 10.20.193.27:0/2361254777 shutdown_connections 2026-02-13T02:20:26.621 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.612+0000 7fad9a87c6c0 1 -- 10.20.193.27:0/2361254777 wait complete. 2026-02-13T02:20:26.621 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.612+0000 7fad9a87c6c0 1 Processor -- start 2026-02-13T02:20:26.621 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.613+0000 7fad9a87c6c0 1 -- start start 2026-02-13T02:20:26.621 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.613+0000 7fad9a87c6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fad9410b270 0x7fad940f4950 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.621 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.613+0000 7fad9a87c6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fad9410c3f0 0x7fad9416d710 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.621 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.613+0000 7fad9a87c6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fad9410cda0 0x7fad940f5190 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.622 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.613+0000 7fad9a87c6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fad940f5de0 con 0x7fad9410cda0 2026-02-13T02:20:26.622 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.613+0000 7fad98dee6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fad9410cda0 0x7fad940f5190 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.622 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.613+0000 7fad98dee6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fad9410cda0 0x7fad940f5190 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:43842/0 (socket says 10.20.193.27:43842) 2026-02-13T02:20:26.622 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.613+0000 7fad98dee6c0 1 -- 10.20.193.27:0/4254534103 learned_addr learned my addr 10.20.193.27:0/4254534103 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:20:26.622 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.613+0000 7fad9a87c6c0 1 -- 10.20.193.27:0/4254534103 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fad940f5f50 con 0x7fad9410b270 2026-02-13T02:20:26.622 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.613+0000 7fad937fe6c0 1 --2- 10.20.193.27:0/4254534103 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fad9410c3f0 0x7fad9416d710 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.622 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.613+0000 7fad93fff6c0 1 --2- 10.20.193.27:0/4254534103 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fad9410b270 0x7fad940f4950 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.622 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.613+0000 7fad9a87c6c0 1 -- 10.20.193.27:0/4254534103 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fad940f6110 con 0x7fad9410c3f0 2026-02-13T02:20:26.622 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.614+0000 7f5268ff96c0 1 -- 10.20.193.27:0/357666803 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f525c030670 con 0x7f526c110050 2026-02-13T02:20:26.625 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.614+0000 7fad937fe6c0 1 -- 10.20.193.27:0/4254534103 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fad9410c3f0 msgr2=0x7fad9416d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 10 2026-02-13T02:20:26.625 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.614+0000 7fad937fe6c0 1 -- 10.20.193.27:0/4254534103 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fad9410c3f0 msgr2=0x7fad9416d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-13T02:20:26.625 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.614+0000 7fad937fe6c0 1 --2- 10.20.193.27:0/4254534103 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fad9410c3f0 0x7fad9416d710 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-13T02:20:26.625 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.614+0000 7fad937fe6c0 1 --2- 10.20.193.27:0/4254534103 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fad9410c3f0 0x7fad9416d710 unknown :-1 s=START_CONNECT pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-13T02:20:26.625 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.614+0000 7fad93fff6c0 1 -- 10.20.193.27:0/4254534103 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fad9410b270 msgr2=0x7fad940f4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 9 2026-02-13T02:20:26.625 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.614+0000 7fad93fff6c0 1 -- 10.20.193.27:0/4254534103 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fad9410b270 msgr2=0x7fad940f4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-13T02:20:26.625 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.614+0000 7fad93fff6c0 1 --2- 10.20.193.27:0/4254534103 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fad9410b270 0x7fad940f4950 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-13T02:20:26.625 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.614+0000 7fad93fff6c0 1 --2- 10.20.193.27:0/4254534103 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fad9410b270 0x7fad940f4950 unknown :-1 s=START_CONNECT pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-13T02:20:26.625 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.614+0000 7fad98dee6c0 1 -- 10.20.193.27:0/4254534103 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fad9410c3f0 msgr2=0x7fad9416d710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-13T02:20:26.625 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.614+0000 7fad98dee6c0 1 --2- 10.20.193.27:0/4254534103 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fad9410c3f0 0x7fad9416d710 unknown :-1 s=START_CONNECT pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.626 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.614+0000 7fad98dee6c0 1 -- 10.20.193.27:0/4254534103 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fad9410b270 msgr2=0x7fad940f4950 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-13T02:20:26.626 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.614+0000 7fad98dee6c0 1 --2- 10.20.193.27:0/4254534103 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fad9410b270 0x7fad940f4950 unknown :-1 s=START_CONNECT pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.626 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.614+0000 7fad98dee6c0 1 -- 10.20.193.27:0/4254534103 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad8800a5b0 con 0x7fad9410cda0 2026-02-13T02:20:26.626 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.614+0000 7fad98dee6c0 1 --2- 10.20.193.27:0/4254534103 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fad9410cda0 0x7fad940f5190 secure :-1 s=READY pgs=122 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fad880027d0 tx=0x7fad88003810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.626 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.614+0000 7fad917fa6c0 1 -- 10.20.193.27:0/4254534103 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fad88003b20 con 0x7fad9410cda0 2026-02-13T02:20:26.626 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.614+0000 7fad917fa6c0 1 -- 10.20.193.27:0/4254534103 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fad88003c80 con 0x7fad9410cda0 2026-02-13T02:20:26.626 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.615+0000 7fad917fa6c0 1 -- 10.20.193.27:0/4254534103 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fad88008b20 con 0x7fad9410cda0 2026-02-13T02:20:26.626 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.617+0000 7fad9a87c6c0 1 -- 10.20.193.27:0/4254534103 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad9405ecd0 con 0x7fad9410cda0 2026-02-13T02:20:26.626 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.617+0000 7fad9a87c6c0 1 -- 10.20.193.27:0/4254534103 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad9405f0a0 con 0x7fad9410cda0 2026-02-13T02:20:26.626 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.617+0000 7fad9a87c6c0 1 -- 10.20.193.27:0/4254534103 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad940f6a10 con 0x7fad9410cda0 2026-02-13T02:20:26.626 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.619+0000 7fad917fa6c0 1 -- 10.20.193.27:0/4254534103 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 8) ==== 52814+0+0 (secure 0 0 0) 0x7fad88008260 con 0x7fad9410cda0 2026-02-13T02:20:26.626 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.620+0000 7fad917fa6c0 1 --2- 10.20.193.27:0/4254534103 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fad6803f870 0x7fad6805fd20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.626 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.620+0000 7fad917fa6c0 1 -- 10.20.193.27:0/4254534103 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 935+0+0 (secure 0 0 0) 0x7fad8807acb0 con 0x7fad9410cda0 2026-02-13T02:20:26.626 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.620+0000 7fad93fff6c0 1 --2- 10.20.193.27:0/4254534103 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fad6803f870 0x7fad6805fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=6 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.627 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.620+0000 7fad93fff6c0 1 --2- 10.20.193.27:0/4254534103 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fad6803f870 0x7fad6805fd20 secure :-1 s=READY pgs=65 gs=6 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fad80004750 tx=0x7fad80005ac0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.627 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.623+0000 7fad917fa6c0 1 -- 10.20.193.27:0/4254534103 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fad8803cd90 con 0x7fad9410cda0 2026-02-13T02:20:26.629 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.627+0000 7f48a37fe6c0 1 -- 10.20.193.27:0/2718718715 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f4894005b40 con 0x7f48d4110050 2026-02-13T02:20:26.631 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.629+0000 7f859aa516c0 1 -- 10.20.193.27:0/4073212428 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f8594065bb0 con 0x7f8594110050 2026-02-13T02:20:26.631 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.630+0000 7f48c9ffb6c0 1 -- 10.20.193.27:0/2718718715 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v20)=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f48bc0088d0 con 0x7f48d4110050 2026-02-13T02:20:26.634 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.632+0000 7f85917fa6c0 1 -- 10.20.193.27:0/4073212428 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v21)=0 v21) ==== 122+0+0 (secure 0 0 0) 0x7f8594065bb0 con 0x7f8594110050 2026-02-13T02:20:26.634 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.632+0000 7f48a37fe6c0 1 -- 10.20.193.27:0/2718718715 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f48b003f8e0 msgr2=0x7f48b005fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.634 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.632+0000 7f48a37fe6c0 1 --2- 10.20.193.27:0/2718718715 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f48b003f8e0 0x7f48b005fd90 secure :-1 s=READY pgs=57 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f48c40023e0 tx=0x7f48c402ff90 comp rx=0 tx=0).stop 2026-02-13T02:20:26.634 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.632+0000 7f1870a986c0 1 -- 10.20.193.27:0/2232626932 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f186c10c840 con 0x7f186c0554b0 2026-02-13T02:20:26.635 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.632+0000 7f48a37fe6c0 1 -- 10.20.193.27:0/2718718715 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f48d4110050 msgr2=0x7f48d411b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.635 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.632+0000 7f48a37fe6c0 1 --2- 10.20.193.27:0/2718718715 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f48d4110050 0x7f48d411b150 secure :-1 s=READY pgs=108 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f48bc005dc0 tx=0x7f48bc00bbd0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.635 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.633+0000 7f48a37fe6c0 1 -- 10.20.193.27:0/2718718715 shutdown_connections 2026-02-13T02:20:26.635 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.633+0000 7f48a37fe6c0 1 --2- 10.20.193.27:0/2718718715 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f48b003f8e0 0x7f48b005fd90 unknown :-1 s=CLOSED pgs=57 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.635 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.633+0000 7f48a37fe6c0 1 --2- 10.20.193.27:0/2718718715 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f48d4110a00 0x7f48d411bd40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.635 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.633+0000 7f48a37fe6c0 1 --2- 10.20.193.27:0/2718718715 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f48d4110050 0x7f48d411b150 unknown :-1 s=CLOSED pgs=108 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.635 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.633+0000 7f48a37fe6c0 1 --2- 10.20.193.27:0/2718718715 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f48d410eed0 0x7f48d411ab00 unknown :-1 s=CLOSED pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.635 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.633+0000 7f48a37fe6c0 1 -- 10.20.193.27:0/2718718715 >> 10.20.193.27:0/2718718715 conn(0x7f48d40ec5f0 msgr2=0x7f48d4111360 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.635 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.633+0000 7f48a37fe6c0 1 -- 10.20.193.27:0/2718718715 shutdown_connections 2026-02-13T02:20:26.636 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.633+0000 7f48a37fe6c0 1 -- 10.20.193.27:0/2718718715 wait complete. 2026-02-13T02:20:26.636 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.634+0000 7fe633cfd6c0 1 -- 10.20.193.27:0/4207596260 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe62c1109f0 msgr2=0x7fe62c171380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.637 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.634+0000 7fe633cfd6c0 1 --2- 10.20.193.27:0/4207596260 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe62c1109f0 0x7fe62c171380 secure :-1 s=READY pgs=123 gs=1 cs=0 l=1 c_cookie=17655496997025889211 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe62800aa40 tx=0x7fe62802fe00 comp rx=0 tx=0).stop 2026-02-13T02:20:26.637 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.634+0000 7fe633cfd6c0 1 -- 10.20.193.27:0/4207596260 shutdown_connections 2026-02-13T02:20:26.637 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.634+0000 7fe633cfd6c0 1 --2- 10.20.193.27:0/4207596260 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe62c1109f0 0x7fe62c171380 unknown :-1 s=CLOSED pgs=123 gs=1 cs=0 l=1 c_cookie=17655496997025889211 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.637 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.634+0000 7fe633cfd6c0 1 --2- 10.20.193.27:0/4207596260 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe62c110040 0x7fe62c1104b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.637 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.634+0000 7fe633cfd6c0 1 --2- 10.20.193.27:0/4207596260 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe62c10eec0 0x7fe62c10f2b0 unknown :-1 s=CLOSED pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.637 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.634+0000 7fe633cfd6c0 1 -- 10.20.193.27:0/4207596260 >> 10.20.193.27:0/4207596260 conn(0x7fe62c0ec5e0 msgr2=0x7fe62c10c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.637 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.634+0000 7fe633cfd6c0 1 -- 10.20.193.27:0/4207596260 shutdown_connections 2026-02-13T02:20:26.637 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.634+0000 7fe633cfd6c0 1 -- 10.20.193.27:0/4207596260 wait complete. 2026-02-13T02:20:26.637 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.634+0000 7fe633cfd6c0 1 Processor -- start 2026-02-13T02:20:26.637 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.634+0000 7fe633cfd6c0 1 -- start start 2026-02-13T02:20:26.637 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.635+0000 7fe633cfd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe62c10eec0 0x7fe62c0f83d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.638 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.635+0000 7fe633cfd6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe62c110040 0x7fe62c171360 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.638 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.635+0000 7f18537fe6c0 1 -- 10.20.193.27:0/2232626932 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v22)=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7f186000d550 con 0x7f186c0554b0 2026-02-13T02:20:26.639 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.635+0000 7fe631a6e6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe62c10eec0 0x7fe62c0f83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.639 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.635+0000 7fe631a6e6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe62c10eec0 0x7fe62c0f83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:43850/0 (socket says 10.20.193.27:43850) 2026-02-13T02:20:26.639 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.635+0000 7fe633cfd6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe62c1109f0 0x7fe62c0f8c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.639 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.635+0000 7fe633cfd6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c0f9860 con 0x7fe62c10eec0 2026-02-13T02:20:26.639 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.635+0000 7fe633cfd6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c0f99d0 con 0x7fe62c110040 2026-02-13T02:20:26.639 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.635+0000 7fe633cfd6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c0f9b90 con 0x7fe62c1109f0 2026-02-13T02:20:26.639 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.635+0000 7fe631a6e6c0 1 -- 10.20.193.27:0/3514685420 learned_addr learned my addr 10.20.193.27:0/3514685420 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:20:26.639 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.635+0000 7fe63126d6c0 1 --2- 10.20.193.27:0/3514685420 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe62c110040 0x7fe62c171360 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.639 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.635+0000 7fe63126d6c0 1 -- 10.20.193.27:0/3514685420 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe62c110040 msgr2=0x7fe62c171360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 10 2026-02-13T02:20:26.639 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.635+0000 7fe63126d6c0 1 -- 10.20.193.27:0/3514685420 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe62c110040 msgr2=0x7fe62c171360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-13T02:20:26.640 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.635+0000 7fe63126d6c0 1 --2- 10.20.193.27:0/3514685420 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe62c110040 0x7fe62c171360 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-13T02:20:26.640 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.635+0000 7fe63126d6c0 1 --2- 10.20.193.27:0/3514685420 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe62c110040 0x7fe62c171360 unknown :-1 s=START_CONNECT pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-13T02:20:26.640 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.635+0000 7fe631a6e6c0 1 -- 10.20.193.27:0/3514685420 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe62c1109f0 msgr2=0x7fe62c0f8c10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2026-02-13T02:20:26.640 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.635+0000 7fe631a6e6c0 1 --2- 10.20.193.27:0/3514685420 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe62c1109f0 0x7fe62c0f8c10 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.640 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.635+0000 7fe631a6e6c0 1 -- 10.20.193.27:0/3514685420 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe62c110040 msgr2=0x7fe62c171360 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-13T02:20:26.640 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.635+0000 7fe631a6e6c0 1 --2- 10.20.193.27:0/3514685420 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe62c110040 0x7fe62c171360 unknown :-1 s=START_CONNECT pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.640 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.635+0000 7fe631a6e6c0 1 -- 10.20.193.27:0/3514685420 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62800a660 con 0x7fe62c10eec0 2026-02-13T02:20:26.640 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.635+0000 7fe631a6e6c0 1 --2- 10.20.193.27:0/3514685420 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe62c10eec0 0x7fe62c0f83d0 secure :-1 s=READY pgs=124 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe61c005dc0 tx=0x7fe61c00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.641 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.635+0000 7f859aa516c0 1 -- 10.20.193.27:0/4073212428 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f856803f940 msgr2=0x7f856805fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.641 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.635+0000 7f859aa516c0 1 --2- 10.20.193.27:0/4073212428 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f856803f940 0x7f856805fdf0 secure :-1 s=READY pgs=58 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8580005fd0 tx=0x7f8580005910 comp rx=0 tx=0).stop 2026-02-13T02:20:26.641 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.635+0000 7fe61affd6c0 1 -- 10.20.193.27:0/3514685420 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe61c004250 con 0x7fe62c10eec0 2026-02-13T02:20:26.642 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.636+0000 7fe61affd6c0 1 -- 10.20.193.27:0/3514685420 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fe61c004d20 con 0x7fe62c10eec0 2026-02-13T02:20:26.642 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.635+0000 7f859aa516c0 1 -- 10.20.193.27:0/4073212428 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8594110050 msgr2=0x7f8594171390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.642 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.635+0000 7f859aa516c0 1 --2- 10.20.193.27:0/4073212428 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8594110050 0x7f8594171390 secure :-1 s=READY pgs=109 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8584005b00 tx=0x7f8584003ae0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.642 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.636+0000 7fe633cfd6c0 1 -- 10.20.193.27:0/3514685420 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe62c0f9e10 con 0x7fe62c10eec0 2026-02-13T02:20:26.643 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.636+0000 7f859aa516c0 1 -- 10.20.193.27:0/4073212428 shutdown_connections 2026-02-13T02:20:26.643 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.636+0000 7f859aa516c0 1 --2- 10.20.193.27:0/4073212428 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f856803f940 0x7f856805fdf0 unknown :-1 s=CLOSED pgs=58 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.643 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.636+0000 7f859aa516c0 1 --2- 10.20.193.27:0/4073212428 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8594110a00 0x7f8594101360 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.643 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.636+0000 7f859aa516c0 1 --2- 10.20.193.27:0/4073212428 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8594110050 0x7f8594171390 unknown :-1 s=CLOSED pgs=109 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.643 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.636+0000 7f859aa516c0 1 --2- 10.20.193.27:0/4073212428 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f859410eed0 0x7f8594100b00 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.643 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.636+0000 7f859aa516c0 1 -- 10.20.193.27:0/4073212428 >> 10.20.193.27:0/4073212428 conn(0x7f85940ec5f0 msgr2=0x7f85940561d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.644 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.636+0000 7fe61affd6c0 1 -- 10.20.193.27:0/3514685420 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe61c004250 con 0x7fe62c10eec0 2026-02-13T02:20:26.644 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.636+0000 7fe633cfd6c0 1 -- 10.20.193.27:0/3514685420 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe62c0fa340 con 0x7fe62c10eec0 2026-02-13T02:20:26.644 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.637+0000 7fe633cfd6c0 1 -- 10.20.193.27:0/3514685420 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe62c1752b0 con 0x7fe62c10eec0 2026-02-13T02:20:26.644 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.637+0000 7fe61affd6c0 1 -- 10.20.193.27:0/3514685420 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 8) ==== 52814+0+0 (secure 0 0 0) 0x7fe61c004780 con 0x7fe62c10eec0 2026-02-13T02:20:26.644 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.637+0000 7fe61affd6c0 1 --2- 10.20.193.27:0/3514685420 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe60803f950 0x7fe60805fe00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.644 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.638+0000 7fe61affd6c0 1 -- 10.20.193.27:0/3514685420 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 935+0+0 (secure 0 0 0) 0x7fe61c050f10 con 0x7fe62c10eec0 2026-02-13T02:20:26.645 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.638+0000 7f859aa516c0 1 -- 10.20.193.27:0/4073212428 shutdown_connections 2026-02-13T02:20:26.645 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.638+0000 7f859aa516c0 1 -- 10.20.193.27:0/4073212428 wait complete. 2026-02-13T02:20:26.645 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.638+0000 7fe63126d6c0 1 --2- 10.20.193.27:0/3514685420 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe60803f950 0x7fe60805fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.645 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.638+0000 7f1870a986c0 1 -- 10.20.193.27:0/2232626932 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f184403f8f0 msgr2=0x7f184405fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.646 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.638+0000 7f1870a986c0 1 --2- 10.20.193.27:0/2232626932 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f184403f8f0 0x7f184405fda0 secure :-1 s=READY pgs=59 gs=6 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f185400aa10 tx=0x7f18540023a0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.646 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.638+0000 7f1870a986c0 1 -- 10.20.193.27:0/2232626932 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f186c0554b0 msgr2=0x7f186c0ef2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.646 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.638+0000 7f1870a986c0 1 --2- 10.20.193.27:0/2232626932 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f186c0554b0 0x7f186c0ef2b0 secure :-1 s=READY pgs=111 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f186000b6c0 tx=0x7f186000bb90 comp rx=0 tx=0).stop 2026-02-13T02:20:26.646 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.638+0000 7fe63126d6c0 1 --2- 10.20.193.27:0/3514685420 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe60803f950 0x7fe60805fe00 secure :-1 s=READY pgs=66 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe620005980 tx=0x7fe620005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.647 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.638+0000 7f1870a986c0 1 -- 10.20.193.27:0/2232626932 shutdown_connections 2026-02-13T02:20:26.647 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.638+0000 7f1870a986c0 1 --2- 10.20.193.27:0/2232626932 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f184403f8f0 0x7f184405fda0 unknown :-1 s=CLOSED pgs=59 gs=6 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.647 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.638+0000 7f1870a986c0 1 --2- 10.20.193.27:0/2232626932 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f186c14d2c0 0x7f186c0f04f0 unknown :-1 s=CLOSED pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.647 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.638+0000 7f1870a986c0 1 --2- 10.20.193.27:0/2232626932 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f186c10ae10 0x7f186c0ef900 unknown :-1 s=CLOSED pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.647 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.638+0000 7f1870a986c0 1 --2- 10.20.193.27:0/2232626932 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f186c0554b0 0x7f186c0ef2b0 unknown :-1 s=CLOSED pgs=111 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.647 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.638+0000 7f1870a986c0 1 -- 10.20.193.27:0/2232626932 >> 10.20.193.27:0/2232626932 conn(0x7f186c0e8510 msgr2=0x7f186c05c110 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.647 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.638+0000 7f1870a986c0 1 -- 10.20.193.27:0/2232626932 shutdown_connections 2026-02-13T02:20:26.647 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.638+0000 7f1870a986c0 1 -- 10.20.193.27:0/2232626932 wait complete. 2026-02-13T02:20:26.647 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.640+0000 7fe61affd6c0 1 -- 10.20.193.27:0/3514685420 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fe61c00f040 con 0x7fe62c10eec0 2026-02-13T02:20:26.648 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.640+0000 7fb5f871c6c0 1 -- 10.20.193.27:0/416812117 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7fb5f010f2a0 con 0x7fb5f010eed0 2026-02-13T02:20:26.648 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.644+0000 7fb5d77fe6c0 1 -- 10.20.193.27:0/416812117 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v23)=0 v23) ==== 118+0+0 (secure 0 0 0) 0x7fb5e0057040 con 0x7fb5f010eed0 2026-02-13T02:20:26.651 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.648+0000 7fb5f871c6c0 1 -- 10.20.193.27:0/416812117 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb5cc03f8f0 msgr2=0x7fb5cc05fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.651 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.649+0000 7fb5f871c6c0 1 --2- 10.20.193.27:0/416812117 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb5cc03f8f0 0x7fb5cc05fda0 secure :-1 s=READY pgs=60 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb5e4005980 tx=0x7fb5e4005910 comp rx=0 tx=0).stop 2026-02-13T02:20:26.651 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.649+0000 7fb5f871c6c0 1 -- 10.20.193.27:0/416812117 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb5f010eed0 msgr2=0x7fb5f00edea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.651 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.649+0000 7fb5f871c6c0 1 --2- 10.20.193.27:0/416812117 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb5f010eed0 0x7fb5f00edea0 secure :-1 s=READY pgs=112 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb5e00098c0 tx=0x7fb5e0054a70 comp rx=0 tx=0).stop 2026-02-13T02:20:26.651 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.649+0000 7fb5f871c6c0 1 -- 10.20.193.27:0/416812117 shutdown_connections 2026-02-13T02:20:26.651 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.649+0000 7fb5f871c6c0 1 --2- 10.20.193.27:0/416812117 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb5cc03f8f0 0x7fb5cc05fda0 unknown :-1 s=CLOSED pgs=60 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.651 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.649+0000 7fb5f871c6c0 1 --2- 10.20.193.27:0/416812117 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb5f0110a00 0x7fb5f00ef0e0 unknown :-1 s=CLOSED pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.651 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.649+0000 7fb5f871c6c0 1 --2- 10.20.193.27:0/416812117 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb5f0110050 0x7fb5f00ee4f0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.651 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.649+0000 7fb5f871c6c0 1 --2- 10.20.193.27:0/416812117 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb5f010eed0 0x7fb5f00edea0 unknown :-1 s=CLOSED pgs=112 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.652 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.649+0000 7fb5f871c6c0 1 -- 10.20.193.27:0/416812117 >> 10.20.193.27:0/416812117 conn(0x7fb5f00ec5f0 msgr2=0x7fb5f00638f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.652 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.649+0000 7fb5f871c6c0 1 -- 10.20.193.27:0/416812117 shutdown_connections 2026-02-13T02:20:26.652 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.649+0000 7fb5f871c6c0 1 -- 10.20.193.27:0/416812117 wait complete. 2026-02-13T02:20:26.656 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.655+0000 7f21af7a96c0 1 -- 10.20.193.27:0/3166427517 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f21a810ddd0 msgr2=0x7f21a810e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.656 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.655+0000 7f21af7a96c0 1 --2- 10.20.193.27:0/3166427517 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f21a810ddd0 0x7f21a810e240 secure :-1 s=READY pgs=125 gs=1 cs=0 l=1 c_cookie=16361433852858183633 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f219c0098a0 tx=0x7f219c02ec40 comp rx=0 tx=0).stop 2026-02-13T02:20:26.657 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.655+0000 7f21af7a96c0 1 -- 10.20.193.27:0/3166427517 shutdown_connections 2026-02-13T02:20:26.657 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.655+0000 7f21af7a96c0 1 --2- 10.20.193.27:0/3166427517 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f21a810e780 0x7f21a816f110 unknown :-1 s=CLOSED pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.657 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.655+0000 7f21af7a96c0 1 --2- 10.20.193.27:0/3166427517 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f21a810ddd0 0x7f21a810e240 unknown :-1 s=CLOSED pgs=125 gs=1 cs=0 l=1 c_cookie=16361433852858183633 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.657 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.655+0000 7f21af7a96c0 1 --2- 10.20.193.27:0/3166427517 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f21a810cc50 0x7f21a810d040 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.657 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.655+0000 7f21af7a96c0 1 -- 10.20.193.27:0/3166427517 >> 10.20.193.27:0/3166427517 conn(0x7f21a80ea370 msgr2=0x7f21a810a780 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.657 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.655+0000 7f21af7a96c0 1 -- 10.20.193.27:0/3166427517 shutdown_connections 2026-02-13T02:20:26.657 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.655+0000 7f21af7a96c0 1 -- 10.20.193.27:0/3166427517 wait complete. 2026-02-13T02:20:26.659 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.655+0000 7f21af7a96c0 1 Processor -- start 2026-02-13T02:20:26.659 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.655+0000 7f21af7a96c0 1 -- start start 2026-02-13T02:20:26.659 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.655+0000 7f21af7a96c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f21a810cc50 0x7f21a80f85d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.659 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.656+0000 7f21af7a96c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f21a810ddd0 0x7f21a816f0f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.659 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.656+0000 7f21af7a96c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f21a810e780 0x7f21a80f8e10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.659 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.656+0000 7f21af7a96c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f21a80f9a60 con 0x7f21a810cc50 2026-02-13T02:20:26.659 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.656+0000 7f21af7a96c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f21a80f9bd0 con 0x7f21a810e780 2026-02-13T02:20:26.659 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.656+0000 7f21af7a96c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f21a80f9d90 con 0x7f21a810ddd0 2026-02-13T02:20:26.659 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.656+0000 7f21acd196c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f21a810ddd0 0x7f21a816f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.659 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.656+0000 7f21ad51a6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f21a810cc50 0x7f21a80f85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.659 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.656+0000 7f21ad51a6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f21a810cc50 0x7f21a80f85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:43868/0 (socket says 10.20.193.27:43868) 2026-02-13T02:20:26.660 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.656+0000 7f21acd196c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f21a810ddd0 0x7f21a816f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.107:3300/0 says I am v2:10.20.193.27:41766/0 (socket says 10.20.193.27:41766) 2026-02-13T02:20:26.660 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.656+0000 7f21ad51a6c0 1 -- 10.20.193.27:0/215433190 learned_addr learned my addr 10.20.193.27:0/215433190 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:20:26.660 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.656+0000 7f21add1b6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f21a810e780 0x7f21a80f8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.660 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.656+0000 7f21add1b6c0 1 -- 10.20.193.27:0/215433190 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f21a810e780 msgr2=0x7f21a80f8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 11 2026-02-13T02:20:26.660 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.656+0000 7f21add1b6c0 1 -- 10.20.193.27:0/215433190 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f21a810e780 msgr2=0x7f21a80f8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-13T02:20:26.660 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.656+0000 7f21add1b6c0 1 --2- 10.20.193.27:0/215433190 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f21a810e780 0x7f21a80f8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-13T02:20:26.660 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.656+0000 7f21add1b6c0 1 --2- 10.20.193.27:0/215433190 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f21a810e780 0x7f21a80f8e10 unknown :-1 s=START_CONNECT pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-13T02:20:26.660 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.656+0000 7f21ad51a6c0 1 -- 10.20.193.27:0/215433190 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f21a810ddd0 msgr2=0x7f21a816f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.660 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.656+0000 7f21ad51a6c0 1 --2- 10.20.193.27:0/215433190 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f21a810ddd0 0x7f21a816f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.660 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.656+0000 7f21ad51a6c0 1 -- 10.20.193.27:0/215433190 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f21a810e780 msgr2=0x7f21a80f8e10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-13T02:20:26.660 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.656+0000 7f21ad51a6c0 1 --2- 10.20.193.27:0/215433190 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f21a810e780 0x7f21a80f8e10 unknown :-1 s=START_CONNECT pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.660 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.656+0000 7f21ad51a6c0 1 -- 10.20.193.27:0/215433190 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219c0094c0 con 0x7f21a810cc50 2026-02-13T02:20:26.660 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.656+0000 7f21ad51a6c0 1 --2- 10.20.193.27:0/215433190 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f21a810cc50 0x7f21a80f85d0 secure :-1 s=READY pgs=126 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2198005dc0 tx=0x7f219800bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.661 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.656+0000 7f21967fc6c0 1 -- 10.20.193.27:0/215433190 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2198004250 con 0x7f21a810cc50 2026-02-13T02:20:26.661 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.656+0000 7f21967fc6c0 1 -- 10.20.193.27:0/215433190 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f2198009070 con 0x7f21a810cc50 2026-02-13T02:20:26.661 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.656+0000 7f21af7a96c0 1 -- 10.20.193.27:0/215433190 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21a80fa010 con 0x7f21a810cc50 2026-02-13T02:20:26.661 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.656+0000 7f21967fc6c0 1 -- 10.20.193.27:0/215433190 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f219800cd50 con 0x7f21a810cc50 2026-02-13T02:20:26.661 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.656+0000 7f21af7a96c0 1 -- 10.20.193.27:0/215433190 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21a80fa490 con 0x7f21a810cc50 2026-02-13T02:20:26.662 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.657+0000 7f21967fc6c0 1 -- 10.20.193.27:0/215433190 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 8) ==== 52814+0+0 (secure 0 0 0) 0x7f21980028c0 con 0x7f21a810cc50 2026-02-13T02:20:26.662 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.657+0000 7f21967fc6c0 1 --2- 10.20.193.27:0/215433190 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f218403f8f0 0x7f218405fda0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.662 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.657+0000 7f21967fc6c0 1 -- 10.20.193.27:0/215433190 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 935+0+0 (secure 0 0 0) 0x7f2198051320 con 0x7f21a810cc50 2026-02-13T02:20:26.662 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.658+0000 7f21af7a96c0 1 -- 10.20.193.27:0/215433190 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21a8173010 con 0x7f21a810cc50 2026-02-13T02:20:26.662 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.658+0000 7f21acd196c0 1 --2- 10.20.193.27:0/215433190 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f218403f8f0 0x7f218405fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.662 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.660+0000 7f21acd196c0 1 --2- 10.20.193.27:0/215433190 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f218403f8f0 0x7f218405fda0 secure :-1 s=READY pgs=67 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f219c02f940 tx=0x7f219c0023a0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.664 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '707dbb29-0882-11f1-9262-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='trial027') 2026-02-13T02:20:26.664 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.662+0000 7f21967fc6c0 1 -- 10.20.193.27:0/215433190 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f2198007a70 con 0x7f21a810cc50 2026-02-13T02:20:26.666 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.664+0000 7f96f45cc6c0 1 -- 10.20.193.27:0/2894620547 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f96ec10eed0 msgr2=0x7f96ec10f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.666 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.664+0000 7f96f45cc6c0 1 --2- 10.20.193.27:0/2894620547 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f96ec10eed0 0x7f96ec10f2c0 secure :-1 s=READY pgs=127 gs=1 cs=0 l=1 c_cookie=6891400807316773231 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f96dc0098a0 tx=0x7f96dc02ec40 comp rx=0 tx=0).stop 2026-02-13T02:20:26.667 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.664+0000 7f96f45cc6c0 1 -- 10.20.193.27:0/2894620547 shutdown_connections 2026-02-13T02:20:26.667 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.664+0000 7f96f45cc6c0 1 --2- 10.20.193.27:0/2894620547 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f96ec110a00 0x7f96ec171390 unknown :-1 s=CLOSED pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.667 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.664+0000 7f96f45cc6c0 1 --2- 10.20.193.27:0/2894620547 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f96ec110050 0x7f96ec1104c0 unknown :-1 s=CLOSED pgs=0 gs=5 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.667 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.664+0000 7f96f45cc6c0 1 --2- 10.20.193.27:0/2894620547 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f96ec10eed0 0x7f96ec10f2c0 unknown :-1 s=CLOSED pgs=127 gs=1 cs=0 l=1 c_cookie=6891400807316773231 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.667 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.664+0000 7f96f45cc6c0 1 -- 10.20.193.27:0/2894620547 >> 10.20.193.27:0/2894620547 conn(0x7f96ec0ec5f0 msgr2=0x7f96ec10ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.667 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.664+0000 7f96f45cc6c0 1 -- 10.20.193.27:0/2894620547 shutdown_connections 2026-02-13T02:20:26.667 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.664+0000 7f96f45cc6c0 1 -- 10.20.193.27:0/2894620547 wait complete. 2026-02-13T02:20:26.669 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.665+0000 7f96f45cc6c0 1 Processor -- start 2026-02-13T02:20:26.670 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.665+0000 7f96f45cc6c0 1 -- start start 2026-02-13T02:20:26.670 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.666+0000 7f96f45cc6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f96ec10eed0 0x7f96ec0f85b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.670 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.666+0000 7f96f23416c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f96ec10eed0 0x7f96ec0f85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.670 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.666+0000 7f96f23416c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f96ec10eed0 0x7f96ec0f85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:43882/0 (socket says 10.20.193.27:43882) 2026-02-13T02:20:26.670 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.666+0000 7f96f23416c0 1 -- 10.20.193.27:0/2203054161 learned_addr learned my addr 10.20.193.27:0/2203054161 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:20:26.670 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.666+0000 7f96f45cc6c0 1 --2- 10.20.193.27:0/2203054161 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f96ec110050 0x7f96ec171390 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.670 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.666+0000 7f96f45cc6c0 1 --2- 10.20.193.27:0/2203054161 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f96ec110a00 0x7f96ec0f8e10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.670 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.666+0000 7f96f45cc6c0 1 -- 10.20.193.27:0/2203054161 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec0f9a40 con 0x7f96ec10eed0 2026-02-13T02:20:26.670 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.666+0000 7f96f45cc6c0 1 -- 10.20.193.27:0/2203054161 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec0f9bb0 con 0x7f96ec110a00 2026-02-13T02:20:26.670 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.666+0000 7f96f45cc6c0 1 -- 10.20.193.27:0/2203054161 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec0f9d70 con 0x7f96ec110050 2026-02-13T02:20:26.671 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.666+0000 7f96f1b406c0 1 --2- 10.20.193.27:0/2203054161 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f96ec110050 0x7f96ec171390 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.671 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.666+0000 7f96f2b426c0 1 --2- 10.20.193.27:0/2203054161 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f96ec110a00 0x7f96ec0f8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.671 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.666+0000 7f96f1b406c0 1 -- 10.20.193.27:0/2203054161 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f96ec110050 msgr2=0x7f96ec171390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 10 2026-02-13T02:20:26.671 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.666+0000 7f96f1b406c0 1 -- 10.20.193.27:0/2203054161 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f96ec110050 msgr2=0x7f96ec171390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-13T02:20:26.671 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.666+0000 7f96f1b406c0 1 --2- 10.20.193.27:0/2203054161 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f96ec110050 0x7f96ec171390 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-13T02:20:26.671 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.666+0000 7f96f1b406c0 1 --2- 10.20.193.27:0/2203054161 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f96ec110050 0x7f96ec171390 unknown :-1 s=START_CONNECT pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-13T02:20:26.671 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.667+0000 7f96f2b426c0 1 -- 10.20.193.27:0/2203054161 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f96ec110a00 msgr2=0x7f96ec0f8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 11 2026-02-13T02:20:26.671 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.667+0000 7f96f2b426c0 1 -- 10.20.193.27:0/2203054161 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f96ec110a00 msgr2=0x7f96ec0f8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-13T02:20:26.671 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.667+0000 7f96f2b426c0 1 --2- 10.20.193.27:0/2203054161 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f96ec110a00 0x7f96ec0f8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-13T02:20:26.671 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.667+0000 7f96f2b426c0 1 --2- 10.20.193.27:0/2203054161 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f96ec110a00 0x7f96ec0f8e10 unknown :-1 s=START_CONNECT pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-13T02:20:26.671 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.667+0000 7f96f23416c0 1 -- 10.20.193.27:0/2203054161 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f96ec110050 msgr2=0x7f96ec171390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-13T02:20:26.671 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.667+0000 7f96f23416c0 1 --2- 10.20.193.27:0/2203054161 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f96ec110050 0x7f96ec171390 unknown :-1 s=START_CONNECT pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.671 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.667+0000 7f96f23416c0 1 -- 10.20.193.27:0/2203054161 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f96ec110a00 msgr2=0x7f96ec0f8e10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-13T02:20:26.672 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.667+0000 7f96f23416c0 1 --2- 10.20.193.27:0/2203054161 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f96ec110a00 0x7f96ec0f8e10 unknown :-1 s=START_CONNECT pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.672 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.667+0000 7f96f23416c0 1 -- 10.20.193.27:0/2203054161 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96dc0094c0 con 0x7f96ec10eed0 2026-02-13T02:20:26.672 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.667+0000 7f96f23416c0 1 --2- 10.20.193.27:0/2203054161 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f96ec10eed0 0x7f96ec0f85b0 secure :-1 s=READY pgs=128 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f96dc02f8e0 tx=0x7f96dc002960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.672 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.667+0000 7f96d37fe6c0 1 -- 10.20.193.27:0/2203054161 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f96dc002d70 con 0x7f96ec10eed0 2026-02-13T02:20:26.672 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.668+0000 7f96d37fe6c0 1 -- 10.20.193.27:0/2203054161 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f96dc007680 con 0x7f96ec10eed0 2026-02-13T02:20:26.672 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.668+0000 7f96f45cc6c0 1 -- 10.20.193.27:0/2203054161 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96ec0f9ff0 con 0x7f96ec10eed0 2026-02-13T02:20:26.673 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.668+0000 7f96f45cc6c0 1 -- 10.20.193.27:0/2203054161 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96ec0fa470 con 0x7f96ec10eed0 2026-02-13T02:20:26.673 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.669+0000 7f96d37fe6c0 1 -- 10.20.193.27:0/2203054161 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f96dc0312b0 con 0x7f96ec10eed0 2026-02-13T02:20:26.673 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.669+0000 7f96d37fe6c0 1 -- 10.20.193.27:0/2203054161 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 8) ==== 52814+0+0 (secure 0 0 0) 0x7f96dc031480 con 0x7f96ec10eed0 2026-02-13T02:20:26.673 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.670+0000 7f96d17fa6c0 1 -- 10.20.193.27:0/2203054161 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96b8005310 con 0x7f96ec10eed0 2026-02-13T02:20:26.673 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.670+0000 7f96d37fe6c0 1 --2- 10.20.193.27:0/2203054161 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f96c803f970 0x7f96c805fe20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.673 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.670+0000 7f96d37fe6c0 1 -- 10.20.193.27:0/2203054161 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 935+0+0 (secure 0 0 0) 0x7f96dc076770 con 0x7f96ec10eed0 2026-02-13T02:20:26.674 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.670+0000 7f96f1b406c0 1 --2- 10.20.193.27:0/2203054161 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f96c803f970 0x7f96c805fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=6 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.674 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.671+0000 7f96f1b406c0 1 --2- 10.20.193.27:0/2203054161 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f96c803f970 0x7f96c805fe20 secure :-1 s=READY pgs=68 gs=6 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f96e0004750 tx=0x7f96e0005ac0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.675 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.674+0000 7f96d37fe6c0 1 -- 10.20.193.27:0/2203054161 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f96dc036780 con 0x7f96ec10eed0 2026-02-13T02:20:26.680 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.678+0000 7fc879d096c0 1 -- 10.20.193.27:0/17566515 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc87410c500 msgr2=0x7fc87410ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.680 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.678+0000 7fc879d096c0 1 --2- 10.20.193.27:0/17566515 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc87410c500 0x7fc87410ce40 secure :-1 s=READY pgs=129 gs=3 cs=0 l=1 c_cookie=7788406227513647835 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc868006fd0 tx=0x7fc86802ff80 comp rx=0 tx=0).stop 2026-02-13T02:20:26.680 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.678+0000 7fc879d096c0 1 -- 10.20.193.27:0/17566515 shutdown_connections 2026-02-13T02:20:26.680 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.678+0000 7fc879d096c0 1 --2- 10.20.193.27:0/17566515 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc87410c500 0x7fc87410ce40 unknown :-1 s=CLOSED pgs=129 gs=3 cs=0 l=1 c_cookie=7788406227513647835 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.680 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.678+0000 7fc879d096c0 1 --2- 10.20.193.27:0/17566515 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc87410bb50 0x7fc87410bfc0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.681 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.678+0000 7fc879d096c0 1 --2- 10.20.193.27:0/17566515 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc87410aa90 0x7fc87410ae80 unknown :-1 s=CLOSED pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.681 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.678+0000 7fc879d096c0 1 -- 10.20.193.27:0/17566515 >> 10.20.193.27:0/17566515 conn(0x7fc8740e8510 msgr2=0x7fc874108940 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.681 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.678+0000 7fc879d096c0 1 -- 10.20.193.27:0/17566515 shutdown_connections 2026-02-13T02:20:26.681 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.678+0000 7fc879d096c0 1 -- 10.20.193.27:0/17566515 wait complete. 2026-02-13T02:20:26.681 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.679+0000 7fc879d096c0 1 Processor -- start 2026-02-13T02:20:26.682 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.681+0000 7fc879d096c0 1 -- start start 2026-02-13T02:20:26.682 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.681+0000 7fc879d096c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc87410aa90 0x7fc8740f2dc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.683 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.681+0000 7fc879d096c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc87410bb50 0x7fc8740f3410 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.683 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.681+0000 7fc879d096c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc87410c500 0x7fc8740f4000 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.683 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.681+0000 7fc879d096c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc8740f4790 con 0x7fc87410aa90 2026-02-13T02:20:26.683 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.681+0000 7fc879d096c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc8740f4900 con 0x7fc87410c500 2026-02-13T02:20:26.683 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.681+0000 7fc879d096c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc8740f4ac0 con 0x7fc87410bb50 2026-02-13T02:20:26.684 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.681+0000 7fc8737fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc87410aa90 0x7fc8740f2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.684 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.681+0000 7fc8737fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc87410aa90 0x7fc8740f2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:43906/0 (socket says 10.20.193.27:43906) 2026-02-13T02:20:26.684 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.681+0000 7fc8737fe6c0 1 -- 10.20.193.27:0/1391449949 learned_addr learned my addr 10.20.193.27:0/1391449949 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:20:26.684 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.682+0000 7fc872ffd6c0 1 --2- 10.20.193.27:0/1391449949 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc87410bb50 0x7fc8740f3410 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.684 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.682+0000 7fc8737fe6c0 1 -- 10.20.193.27:0/1391449949 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc87410bb50 msgr2=0x7fc8740f3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.684 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.682+0000 7fc8737fe6c0 1 --2- 10.20.193.27:0/1391449949 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc87410bb50 0x7fc8740f3410 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.684 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.682+0000 7fc8737fe6c0 1 -- 10.20.193.27:0/1391449949 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc87410c500 msgr2=0x7fc8740f4000 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2026-02-13T02:20:26.684 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.682+0000 7fc8737fe6c0 1 --2- 10.20.193.27:0/1391449949 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc87410c500 0x7fc8740f4000 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.684 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.682+0000 7fc8737fe6c0 1 -- 10.20.193.27:0/1391449949 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc86800a5b0 con 0x7fc87410aa90 2026-02-13T02:20:26.685 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.682+0000 7fc8737fe6c0 1 --2- 10.20.193.27:0/1391449949 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc87410aa90 0x7fc8740f2dc0 secure :-1 s=READY pgs=130 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc860005980 tx=0x7fc86000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.685 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.682+0000 7fc870ff96c0 1 -- 10.20.193.27:0/1391449949 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc860004250 con 0x7fc87410aa90 2026-02-13T02:20:26.685 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.682+0000 7fc870ff96c0 1 -- 10.20.193.27:0/1391449949 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc8600043b0 con 0x7fc87410aa90 2026-02-13T02:20:26.685 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.682+0000 7fc870ff96c0 1 -- 10.20.193.27:0/1391449949 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc860011070 con 0x7fc87410aa90 2026-02-13T02:20:26.685 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.682+0000 7fc879d096c0 1 -- 10.20.193.27:0/1391449949 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8740f4da0 con 0x7fc87410aa90 2026-02-13T02:20:26.685 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.682+0000 7fc879d096c0 1 -- 10.20.193.27:0/1391449949 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8740f5280 con 0x7fc87410aa90 2026-02-13T02:20:26.686 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.684+0000 7fc870ff96c0 1 -- 10.20.193.27:0/1391449949 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 8) ==== 52814+0+0 (secure 0 0 0) 0x7fc860011200 con 0x7fc87410aa90 2026-02-13T02:20:26.686 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.684+0000 7fc879d096c0 1 -- 10.20.193.27:0/1391449949 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc874110e70 con 0x7fc87410aa90 2026-02-13T02:20:26.686 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.684+0000 7fc870ff96c0 1 --2- 10.20.193.27:0/1391449949 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc84803f940 0x7fc84805fdf0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.686 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.684+0000 7fc870ff96c0 1 -- 10.20.193.27:0/1391449949 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 935+0+0 (secure 0 0 0) 0x7fc860051210 con 0x7fc87410aa90 2026-02-13T02:20:26.686 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.684+0000 7fc872ffd6c0 1 --2- 10.20.193.27:0/1391449949 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc84803f940 0x7fc84805fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.686 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.685+0000 7fc872ffd6c0 1 --2- 10.20.193.27:0/1391449949 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc84803f940 0x7fc84805fdf0 secure :-1 s=READY pgs=69 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc864005980 tx=0x7fc864005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.689 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.688+0000 7fc870ff96c0 1 -- 10.20.193.27:0/1391449949 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fc860013060 con 0x7fc87410aa90 2026-02-13T02:20:26.690 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:26 trial107 systemd[1]: Started Ceph mgr.z for 707dbb29-0882-11f1-9262-d404e6e7d460. 2026-02-13T02:20:26.690 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:26 trial107 ceph-mgr[28762]: -- 10.20.193.107:0/1775216395 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x55d609c6fb80 con 0x55d609d3b800 2026-02-13T02:20:26.690 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:26 trial107 ceph-mgr[28762]: -- 10.20.193.107:0/1775216395 <== mon.0 v2:10.20.193.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d609c67a00 con 0x55d609d3b800 2026-02-13T02:20:26.691 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.688+0000 7f26bf5696c0 1 -- 10.20.193.27:0/167189678 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f26b810eed0 msgr2=0x7f26b810f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.691 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.688+0000 7f26bf5696c0 1 --2- 10.20.193.27:0/167189678 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f26b810eed0 0x7f26b810f2c0 secure :-1 s=READY pgs=131 gs=3 cs=0 l=1 c_cookie=1646268682632046536 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f26a8009b00 tx=0x7f26a802eeb0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.691 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.689+0000 7f26bf5696c0 1 -- 10.20.193.27:0/167189678 shutdown_connections 2026-02-13T02:20:26.691 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.689+0000 7f26bf5696c0 1 --2- 10.20.193.27:0/167189678 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f26b8110a00 0x7f26b8171390 unknown :-1 s=CLOSED pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.691 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.689+0000 7f26bf5696c0 1 --2- 10.20.193.27:0/167189678 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f26b8110050 0x7f26b81104c0 unknown :-1 s=CLOSED pgs=0 gs=5 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.691 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.689+0000 7f26bf5696c0 1 --2- 10.20.193.27:0/167189678 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f26b810eed0 0x7f26b810f2c0 unknown :-1 s=CLOSED pgs=131 gs=3 cs=0 l=1 c_cookie=1646268682632046536 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.691 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.689+0000 7f26bf5696c0 1 -- 10.20.193.27:0/167189678 >> 10.20.193.27:0/167189678 conn(0x7f26b80ec5f0 msgr2=0x7f26b810ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.691 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.689+0000 7f26bf5696c0 1 -- 10.20.193.27:0/167189678 shutdown_connections 2026-02-13T02:20:26.692 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.689+0000 7f26bf5696c0 1 -- 10.20.193.27:0/167189678 wait complete. 2026-02-13T02:20:26.694 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.690+0000 7f26bf5696c0 1 Processor -- start 2026-02-13T02:20:26.694 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.690+0000 7f26bf5696c0 1 -- start start 2026-02-13T02:20:26.694 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.691+0000 7f26bf5696c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f26b810eed0 0x7f26b8112bb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.694 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.691+0000 7f26bf5696c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f26b8110050 0x7f26b8113200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.694 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.691+0000 7f26bf5696c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f26b8110a00 0x7f26b8113df0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.694 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.691+0000 7f26bcad96c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f26b8110050 0x7f26b8113200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.694 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.691+0000 7f26bcad96c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f26b8110050 0x7f26b8113200 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:43934/0 (socket says 10.20.193.27:43934) 2026-02-13T02:20:26.694 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.691+0000 7f26bd2da6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f26b810eed0 0x7f26b8112bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.694 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.691+0000 7f26bcad96c0 1 -- 10.20.193.27:0/2054756680 learned_addr learned my addr 10.20.193.27:0/2054756680 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:20:26.695 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.691+0000 7f26bf5696c0 1 -- 10.20.193.27:0/2054756680 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f26b8114580 con 0x7f26b8110050 2026-02-13T02:20:26.695 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.691+0000 7f26bf5696c0 1 -- 10.20.193.27:0/2054756680 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f26b81146f0 con 0x7f26b810eed0 2026-02-13T02:20:26.695 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.691+0000 7f26bf5696c0 1 -- 10.20.193.27:0/2054756680 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f26b81148b0 con 0x7f26b8110a00 2026-02-13T02:20:26.695 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.691+0000 7f26bd2da6c0 1 -- 10.20.193.27:0/2054756680 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f26b810eed0 msgr2=0x7f26b8112bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 9 2026-02-13T02:20:26.695 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.691+0000 7f26bd2da6c0 1 -- 10.20.193.27:0/2054756680 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f26b810eed0 msgr2=0x7f26b8112bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-13T02:20:26.695 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.691+0000 7f26bd2da6c0 1 --2- 10.20.193.27:0/2054756680 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f26b810eed0 0x7f26b8112bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-13T02:20:26.695 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.691+0000 7f26bd2da6c0 1 --2- 10.20.193.27:0/2054756680 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f26b810eed0 0x7f26b8112bb0 unknown :-1 s=START_CONNECT pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-13T02:20:26.695 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.692+0000 7f26bcad96c0 1 -- 10.20.193.27:0/2054756680 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f26b8110a00 msgr2=0x7f26b8113df0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2026-02-13T02:20:26.695 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.692+0000 7f26bcad96c0 1 --2- 10.20.193.27:0/2054756680 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f26b8110a00 0x7f26b8113df0 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.695 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.692+0000 7f26bcad96c0 1 -- 10.20.193.27:0/2054756680 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f26b810eed0 msgr2=0x7f26b8112bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-13T02:20:26.695 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.692+0000 7f26bcad96c0 1 --2- 10.20.193.27:0/2054756680 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f26b810eed0 0x7f26b8112bb0 unknown :-1 s=START_CONNECT pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.695 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.692+0000 7f26bcad96c0 1 -- 10.20.193.27:0/2054756680 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a80094c0 con 0x7f26b8110050 2026-02-13T02:20:26.695 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.692+0000 7f26bcad96c0 1 --2- 10.20.193.27:0/2054756680 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f26b8110050 0x7f26b8113200 secure :-1 s=READY pgs=134 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f26ac005dc0 tx=0x7f26ac00bbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.695 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.692+0000 7f26a67fc6c0 1 -- 10.20.193.27:0/2054756680 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f26ac004250 con 0x7f26b8110050 2026-02-13T02:20:26.696 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.692+0000 7f26a67fc6c0 1 -- 10.20.193.27:0/2054756680 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f26ac00be30 con 0x7f26b8110050 2026-02-13T02:20:26.696 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.692+0000 7f26a67fc6c0 1 -- 10.20.193.27:0/2054756680 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f26ac004250 con 0x7f26b8110050 2026-02-13T02:20:26.696 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.692+0000 7f26bf5696c0 1 -- 10.20.193.27:0/2054756680 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b8114b90 con 0x7f26b8110050 2026-02-13T02:20:26.696 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.692+0000 7f26bf5696c0 1 -- 10.20.193.27:0/2054756680 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b81550f0 con 0x7f26b8110050 2026-02-13T02:20:26.697 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.693+0000 7f26a67fc6c0 1 -- 10.20.193.27:0/2054756680 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 8) ==== 52814+0+0 (secure 0 0 0) 0x7f26ac0028c0 con 0x7f26b8110050 2026-02-13T02:20:26.697 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.693+0000 7f26bf5696c0 1 -- 10.20.193.27:0/2054756680 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26b8064e90 con 0x7f26b8110050 2026-02-13T02:20:26.697 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.693+0000 7f26a67fc6c0 1 --2- 10.20.193.27:0/2054756680 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f269403f9b0 0x7f269405fe60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.697 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.693+0000 7f26a67fc6c0 1 -- 10.20.193.27:0/2054756680 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 935+0+0 (secure 0 0 0) 0x7f26ac00ed60 con 0x7f26b8110050 2026-02-13T02:20:26.697 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.694+0000 7f26bd2da6c0 1 --2- 10.20.193.27:0/2054756680 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f269403f9b0 0x7f269405fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.697 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.694+0000 7f26bd2da6c0 1 --2- 10.20.193.27:0/2054756680 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f269403f9b0 0x7f269405fe60 secure :-1 s=READY pgs=70 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f26a80029c0 tx=0x7f26a80023a0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.699 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.697+0000 7f26a67fc6c0 1 -- 10.20.193.27:0/2054756680 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f26ac005490 con 0x7f26b8110050 2026-02-13T02:20:26.703 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.702+0000 7f60445b06c0 1 -- 10.20.193.27:0/28158527 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f603c10ddd0 msgr2=0x7f603c10e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.703 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.702+0000 7f60445b06c0 1 --2- 10.20.193.27:0/28158527 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f603c10ddd0 0x7f603c10e240 secure :-1 s=READY pgs=135 gs=1 cs=0 l=1 c_cookie=12205717951217307681 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6030009970 tx=0x7f603002ed20 comp rx=0 tx=0).stop 2026-02-13T02:20:26.704 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.702+0000 7f60445b06c0 1 -- 10.20.193.27:0/28158527 shutdown_connections 2026-02-13T02:20:26.704 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.702+0000 7f60445b06c0 1 --2- 10.20.193.27:0/28158527 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f603c10e780 0x7f603c16f110 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.704 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.702+0000 7f60445b06c0 1 --2- 10.20.193.27:0/28158527 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f603c10ddd0 0x7f603c10e240 unknown :-1 s=CLOSED pgs=135 gs=1 cs=0 l=1 c_cookie=12205717951217307681 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.704 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.702+0000 7f60445b06c0 1 --2- 10.20.193.27:0/28158527 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f603c10cc50 0x7f603c10d040 unknown :-1 s=CLOSED pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.704 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.702+0000 7f60445b06c0 1 -- 10.20.193.27:0/28158527 >> 10.20.193.27:0/28158527 conn(0x7f603c0ea370 msgr2=0x7f603c10a780 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.704 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.702+0000 7f60445b06c0 1 -- 10.20.193.27:0/28158527 shutdown_connections 2026-02-13T02:20:26.704 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.702+0000 7f60445b06c0 1 -- 10.20.193.27:0/28158527 wait complete. 2026-02-13T02:20:26.706 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.702+0000 7f60445b06c0 1 Processor -- start 2026-02-13T02:20:26.706 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.702+0000 7f60445b06c0 1 -- start start 2026-02-13T02:20:26.706 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.702+0000 7f60445b06c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f603c10cc50 0x7f603c061ea0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.707 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.702+0000 7f60445b06c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f603c10ddd0 0x7f603c16f0f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.707 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.702+0000 7f60445b06c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f603c10e780 0x7f603c0607a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.707 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.702+0000 7f60445b06c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f603c060da0 con 0x7f603c10cc50 2026-02-13T02:20:26.707 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.702+0000 7f60445b06c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f603c060f10 con 0x7f603c10e780 2026-02-13T02:20:26.707 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.702+0000 7f60445b06c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f603c0610d0 con 0x7f603c10ddd0 2026-02-13T02:20:26.707 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.702+0000 7f60423256c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f603c10cc50 0x7f603c061ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.707 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.702+0000 7f60423256c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f603c10cc50 0x7f603c061ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:43946/0 (socket says 10.20.193.27:43946) 2026-02-13T02:20:26.707 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.702+0000 7f60423256c0 1 -- 10.20.193.27:0/2385354456 learned_addr learned my addr 10.20.193.27:0/2385354456 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:20:26.707 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.703+0000 7f6041b246c0 1 --2- 10.20.193.27:0/2385354456 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f603c10ddd0 0x7f603c16f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.707 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.703+0000 7f6042b266c0 1 --2- 10.20.193.27:0/2385354456 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f603c10e780 0x7f603c0607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.707 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.703+0000 7f60423256c0 1 -- 10.20.193.27:0/2385354456 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f603c10ddd0 msgr2=0x7f603c16f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.707 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.703+0000 7f60423256c0 1 --2- 10.20.193.27:0/2385354456 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f603c10ddd0 0x7f603c16f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.708 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.703+0000 7f60423256c0 1 -- 10.20.193.27:0/2385354456 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f603c10e780 msgr2=0x7f603c0607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.708 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.703+0000 7f60423256c0 1 --2- 10.20.193.27:0/2385354456 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f603c10e780 0x7f603c0607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.708 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.703+0000 7f60423256c0 1 -- 10.20.193.27:0/2385354456 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6030009590 con 0x7f603c10cc50 2026-02-13T02:20:26.708 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.703+0000 7f60423256c0 1 --2- 10.20.193.27:0/2385354456 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f603c10cc50 0x7f603c061ea0 secure :-1 s=READY pgs=136 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f602c005dc0 tx=0x7f602c00bbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.708 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.703+0000 7f60237fe6c0 1 -- 10.20.193.27:0/2385354456 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f602c004250 con 0x7f603c10cc50 2026-02-13T02:20:26.708 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.703+0000 7f60445b06c0 1 -- 10.20.193.27:0/2385354456 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f603c0613b0 con 0x7f603c10cc50 2026-02-13T02:20:26.708 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.703+0000 7f60237fe6c0 1 -- 10.20.193.27:0/2385354456 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f602c00be30 con 0x7f603c10cc50 2026-02-13T02:20:26.708 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.703+0000 7f60237fe6c0 1 -- 10.20.193.27:0/2385354456 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f602c0052a0 con 0x7f603c10cc50 2026-02-13T02:20:26.708 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.703+0000 7f60445b06c0 1 -- 10.20.193.27:0/2385354456 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f603c0fa700 con 0x7f603c10cc50 2026-02-13T02:20:26.708 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.704+0000 7f60445b06c0 1 -- 10.20.193.27:0/2385354456 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f603c173040 con 0x7f603c10cc50 2026-02-13T02:20:26.708 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.704+0000 7f60237fe6c0 1 -- 10.20.193.27:0/2385354456 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 8) ==== 52814+0+0 (secure 0 0 0) 0x7f602c005430 con 0x7f603c10cc50 2026-02-13T02:20:26.708 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.704+0000 7f60237fe6c0 1 --2- 10.20.193.27:0/2385354456 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f601803f8f0 0x7f601805fda0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:26.708 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.704+0000 7f60237fe6c0 1 -- 10.20.193.27:0/2385354456 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 935+0+0 (secure 0 0 0) 0x7f602c0514c0 con 0x7f603c10cc50 2026-02-13T02:20:26.709 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.704+0000 7f6041b246c0 1 --2- 10.20.193.27:0/2385354456 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f601803f8f0 0x7f601805fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:26.709 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.705+0000 7f6041b246c0 1 --2- 10.20.193.27:0/2385354456 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f601803f8f0 0x7f601805fda0 secure :-1 s=READY pgs=71 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f603002fa20 tx=0x7f6030005c00 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:26.709 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.708+0000 7f60237fe6c0 1 -- 10.20.193.27:0/2385354456 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f602c0043b0 con 0x7f603c10cc50 2026-02-13T02:20:26.717 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '707dbb29-0882-11f1-9262-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='trial027') 2026-02-13T02:20:26.717 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '707dbb29-0882-11f1-9262-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='trial027') 2026-02-13T02:20:26.718 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '707dbb29-0882-11f1-9262-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='trial027') 2026-02-13T02:20:26.718 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '707dbb29-0882-11f1-9262-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='trial027') 2026-02-13T02:20:26.722 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.720+0000 7fe0d8d196c0 1 -- 10.20.193.27:0/1855195464 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7fe0d410d040 con 0x7fe0d410cc50 2026-02-13T02:20:26.725 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.723+0000 7fe0bb7fe6c0 1 -- 10.20.193.27:0/1855195464 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v24)=0 v24) ==== 111+0+0 (secure 0 0 0) 0x7fe0c8012030 con 0x7fe0d410cc50 2026-02-13T02:20:26.728 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.726+0000 7fe0d8d196c0 1 -- 10.20.193.27:0/1855195464 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe0ac03f870 msgr2=0x7fe0ac05fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.728 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.726+0000 7fe0d8d196c0 1 --2- 10.20.193.27:0/1855195464 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe0ac03f870 0x7fe0ac05fd20 secure :-1 s=READY pgs=62 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe0bc0098c0 tx=0x7fe0bc005ae0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.728 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.726+0000 7fe0d8d196c0 1 -- 10.20.193.27:0/1855195464 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe0d410cc50 msgr2=0x7fe0d40fbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.728 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.726+0000 7fe0d8d196c0 1 --2- 10.20.193.27:0/1855195464 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe0d410cc50 0x7fe0d40fbd40 secure :-1 s=READY pgs=116 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe0c800b750 tx=0x7fe0c800bc20 comp rx=0 tx=0).stop 2026-02-13T02:20:26.729 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.727+0000 7fe0d8d196c0 1 -- 10.20.193.27:0/1855195464 shutdown_connections 2026-02-13T02:20:26.729 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.727+0000 7fe0d8d196c0 1 --2- 10.20.193.27:0/1855195464 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe0ac03f870 0x7fe0ac05fd20 unknown :-1 s=CLOSED pgs=62 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.729 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.727+0000 7fe0d8d196c0 1 --2- 10.20.193.27:0/1855195464 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe0d410e780 0x7fe0d40fc4f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.729 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.727+0000 7fe0d8d196c0 1 --2- 10.20.193.27:0/1855195464 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe0d410ddd0 0x7fe0d416f110 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.729 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.727+0000 7fe0d8d196c0 1 --2- 10.20.193.27:0/1855195464 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe0d410cc50 0x7fe0d40fbd40 unknown :-1 s=CLOSED pgs=116 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.729 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.727+0000 7fe0d8d196c0 1 -- 10.20.193.27:0/1855195464 >> 10.20.193.27:0/1855195464 conn(0x7fe0d40ea370 msgr2=0x7fe0d4173c00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.729 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.727+0000 7fe0d8d196c0 1 -- 10.20.193.27:0/1855195464 shutdown_connections 2026-02-13T02:20:26.729 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.727+0000 7fe0d8d196c0 1 -- 10.20.193.27:0/1855195464 wait complete. 2026-02-13T02:20:26.740 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.739+0000 7fe99b32f6c0 1 -- 10.20.193.27:0/1094763007 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7fe9940008e0 con 0x7fe99410cc50 2026-02-13T02:20:26.743 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.741+0000 7fe9827fc6c0 1 -- 10.20.193.27:0/1094763007 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v25)=0 v25) ==== 119+0+0 (secure 0 0 0) 0x7fe98400d970 con 0x7fe99410cc50 2026-02-13T02:20:26.747 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.745+0000 7fe99b32f6c0 1 -- 10.20.193.27:0/1094763007 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe97003f950 msgr2=0x7fe97005fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.747 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.745+0000 7fe99b32f6c0 1 --2- 10.20.193.27:0/1094763007 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe97003f950 0x7fe97005fe00 secure :-1 s=READY pgs=61 gs=6 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe9880023e0 tx=0x7fe98802ff90 comp rx=0 tx=0).stop 2026-02-13T02:20:26.748 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.745+0000 7fe99b32f6c0 1 -- 10.20.193.27:0/1094763007 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe99410cc50 msgr2=0x7fe994061ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.748 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.745+0000 7fe99b32f6c0 1 --2- 10.20.193.27:0/1094763007 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe99410cc50 0x7fe994061ea0 secure :-1 s=READY pgs=114 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe984005dc0 tx=0x7fe98400bbd0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.748 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.745+0000 7fe99b32f6c0 1 -- 10.20.193.27:0/1094763007 shutdown_connections 2026-02-13T02:20:26.748 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.745+0000 7fe99b32f6c0 1 --2- 10.20.193.27:0/1094763007 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe97003f950 0x7fe97005fe00 unknown :-1 s=CLOSED pgs=61 gs=6 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.748 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.745+0000 7fe99b32f6c0 1 --2- 10.20.193.27:0/1094763007 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe99410e780 0x7fe9940607a0 unknown :-1 s=CLOSED pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.748 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.745+0000 7fe99b32f6c0 1 --2- 10.20.193.27:0/1094763007 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe99410ddd0 0x7fe99416f0f0 unknown :-1 s=CLOSED pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.748 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.745+0000 7fe99b32f6c0 1 --2- 10.20.193.27:0/1094763007 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe99410cc50 0x7fe994061ea0 unknown :-1 s=CLOSED pgs=114 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.748 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.745+0000 7fe99b32f6c0 1 -- 10.20.193.27:0/1094763007 >> 10.20.193.27:0/1094763007 conn(0x7fe9940ea370 msgr2=0x7fe9940ec910 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.748 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.746+0000 7fe99b32f6c0 1 -- 10.20.193.27:0/1094763007 shutdown_connections 2026-02-13T02:20:26.748 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.746+0000 7fe99b32f6c0 1 -- 10.20.193.27:0/1094763007 wait complete. 2026-02-13T02:20:26.749 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.747+0000 7f5271fab6c0 1 -- 10.20.193.27:0/357666803 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f526c063950 con 0x7f526c110050 2026-02-13T02:20:26.752 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.750+0000 7f5268ff96c0 1 -- 10.20.193.27:0/357666803 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v26)=0 v26) ==== 120+0+0 (secure 0 0 0) 0x7f525c031e60 con 0x7f526c110050 2026-02-13T02:20:26.752 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.750+0000 7f3ef2d5c6c0 1 -- 10.20.193.27:0/2445171217 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f3eec0bee20 con 0x7f3eec10eed0 2026-02-13T02:20:26.752 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.751+0000 7f3ee1ffb6c0 1 -- 10.20.193.27:0/2445171217 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v26)=0 v26) ==== 106+0+0 (secure 0 0 0) 0x7f3edc011620 con 0x7f3eec10eed0 2026-02-13T02:20:26.754 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.752+0000 7f5271fab6c0 1 -- 10.20.193.27:0/357666803 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f524003f8f0 msgr2=0x7f524005fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.754 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.752+0000 7f5271fab6c0 1 --2- 10.20.193.27:0/357666803 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f524003f8f0 0x7f524005fda0 secure :-1 s=READY pgs=64 gs=6 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5258006b60 tx=0x7f5258006af0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.755 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.753+0000 7f5271fab6c0 1 -- 10.20.193.27:0/357666803 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f526c110050 msgr2=0x7f526c171390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.755 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.753+0000 7f5271fab6c0 1 --2- 10.20.193.27:0/357666803 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f526c110050 0x7f526c171390 secure :-1 s=READY pgs=121 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f525c009970 tx=0x7f525c0028e0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.755 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.753+0000 7f5271fab6c0 1 -- 10.20.193.27:0/357666803 shutdown_connections 2026-02-13T02:20:26.755 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.753+0000 7f5271fab6c0 1 --2- 10.20.193.27:0/357666803 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f524003f8f0 0x7f524005fda0 unknown :-1 s=CLOSED pgs=64 gs=6 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.755 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.753+0000 7f5271fab6c0 1 --2- 10.20.193.27:0/357666803 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f526c110a00 0x7f526c0ff8b0 unknown :-1 s=CLOSED pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.755 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.753+0000 7f5271fab6c0 1 --2- 10.20.193.27:0/357666803 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f526c110050 0x7f526c171390 unknown :-1 s=CLOSED pgs=121 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.755 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.753+0000 7f5271fab6c0 1 --2- 10.20.193.27:0/357666803 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f526c10eed0 0x7f526c0ff050 unknown :-1 s=CLOSED pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.755 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.753+0000 7f5271fab6c0 1 -- 10.20.193.27:0/357666803 >> 10.20.193.27:0/357666803 conn(0x7f526c0ec5f0 msgr2=0x7f526c0ec9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.755 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.753+0000 7f5271fab6c0 1 -- 10.20.193.27:0/357666803 shutdown_connections 2026-02-13T02:20:26.755 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.753+0000 7f5271fab6c0 1 -- 10.20.193.27:0/357666803 wait complete. 2026-02-13T02:20:26.756 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.754+0000 7f3ef2d5c6c0 1 -- 10.20.193.27:0/2445171217 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3ec803f940 msgr2=0x7f3ec805fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.756 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.754+0000 7f3ef2d5c6c0 1 --2- 10.20.193.27:0/2445171217 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3ec803f940 0x7f3ec805fdf0 secure :-1 s=READY pgs=63 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3ed40098c0 tx=0x7f3ed4005c60 comp rx=0 tx=0).stop 2026-02-13T02:20:26.756 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.754+0000 7f3ef2d5c6c0 1 -- 10.20.193.27:0/2445171217 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3eec10eed0 msgr2=0x7f3eec0fc8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.757 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.754+0000 7f3ef2d5c6c0 1 --2- 10.20.193.27:0/2445171217 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3eec10eed0 0x7f3eec0fc8f0 secure :-1 s=READY pgs=118 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3edc00b710 tx=0x7f3edc00bbe0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.757 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.754+0000 7f3ef2d5c6c0 1 -- 10.20.193.27:0/2445171217 shutdown_connections 2026-02-13T02:20:26.757 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.754+0000 7f3ef2d5c6c0 1 --2- 10.20.193.27:0/2445171217 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3ec803f940 0x7f3ec805fdf0 unknown :-1 s=CLOSED pgs=63 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.757 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.754+0000 7f3ef2d5c6c0 1 --2- 10.20.193.27:0/2445171217 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3eec110a00 0x7f3eec0fd150 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.757 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.754+0000 7f3ef2d5c6c0 1 --2- 10.20.193.27:0/2445171217 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3eec110050 0x7f3eec171390 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.757 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.754+0000 7f3ef2d5c6c0 1 --2- 10.20.193.27:0/2445171217 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3eec10eed0 0x7f3eec0fc8f0 unknown :-1 s=CLOSED pgs=118 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.757 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.754+0000 7f3ef2d5c6c0 1 -- 10.20.193.27:0/2445171217 >> 10.20.193.27:0/2445171217 conn(0x7f3eec0ec5f0 msgr2=0x7f3eec062a70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.757 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.755+0000 7f3ef2d5c6c0 1 -- 10.20.193.27:0/2445171217 shutdown_connections 2026-02-13T02:20:26.757 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.755+0000 7f3ef2d5c6c0 1 -- 10.20.193.27:0/2445171217 wait complete. 2026-02-13T02:20:26.759 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.757+0000 7fad9a87c6c0 1 -- 10.20.193.27:0/4254534103 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fad94170da0 con 0x7fad9410cda0 2026-02-13T02:20:26.760 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.758+0000 7fe633cfd6c0 1 -- 10.20.193.27:0/3514685420 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fe62c174950 con 0x7fe62c10eec0 2026-02-13T02:20:26.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.760+0000 7fad917fa6c0 1 -- 10.20.193.27:0/4254534103 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v27)=0 v27) ==== 120+0+0 (secure 0 0 0) 0x7fad8803c540 con 0x7fad9410cda0 2026-02-13T02:20:26.764 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.763+0000 7fe61affd6c0 1 -- 10.20.193.27:0/3514685420 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v28)=0 v28) ==== 121+0+0 (secure 0 0 0) 0x7fe61c008040 con 0x7fe62c10eec0 2026-02-13T02:20:26.765 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.763+0000 7fad9a87c6c0 1 -- 10.20.193.27:0/4254534103 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fad6803f870 msgr2=0x7fad6805fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.765 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.763+0000 7fad9a87c6c0 1 --2- 10.20.193.27:0/4254534103 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fad6803f870 0x7fad6805fd20 secure :-1 s=READY pgs=65 gs=6 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fad80004750 tx=0x7fad80005ac0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.765 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.763+0000 7fad9a87c6c0 1 -- 10.20.193.27:0/4254534103 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fad9410cda0 msgr2=0x7fad940f5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.765 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.763+0000 7fad9a87c6c0 1 --2- 10.20.193.27:0/4254534103 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fad9410cda0 0x7fad940f5190 secure :-1 s=READY pgs=122 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fad880027d0 tx=0x7fad88003810 comp rx=0 tx=0).stop 2026-02-13T02:20:26.765 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.763+0000 7fad9a87c6c0 1 -- 10.20.193.27:0/4254534103 shutdown_connections 2026-02-13T02:20:26.765 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.763+0000 7fad9a87c6c0 1 --2- 10.20.193.27:0/4254534103 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fad6803f870 0x7fad6805fd20 unknown :-1 s=CLOSED pgs=65 gs=6 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.765 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.763+0000 7fad9a87c6c0 1 --2- 10.20.193.27:0/4254534103 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fad9410cda0 0x7fad940f5190 unknown :-1 s=CLOSED pgs=122 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.765 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.763+0000 7fad9a87c6c0 1 --2- 10.20.193.27:0/4254534103 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fad9410c3f0 0x7fad9416d710 unknown :-1 s=CLOSED pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.766 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.763+0000 7fad9a87c6c0 1 --2- 10.20.193.27:0/4254534103 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fad9410b270 0x7fad940f4950 unknown :-1 s=CLOSED pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.766 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.763+0000 7fad9a87c6c0 1 -- 10.20.193.27:0/4254534103 >> 10.20.193.27:0/4254534103 conn(0x7fad940e8990 msgr2=0x7fad940eaf30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.766 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.763+0000 7fad9a87c6c0 1 -- 10.20.193.27:0/4254534103 shutdown_connections 2026-02-13T02:20:26.766 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.763+0000 7fad9a87c6c0 1 -- 10.20.193.27:0/4254534103 wait complete. 2026-02-13T02:20:26.769 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.767+0000 7fe633cfd6c0 1 -- 10.20.193.27:0/3514685420 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe60803f950 msgr2=0x7fe60805fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.769 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.767+0000 7fe633cfd6c0 1 --2- 10.20.193.27:0/3514685420 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe60803f950 0x7fe60805fe00 secure :-1 s=READY pgs=66 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe620005980 tx=0x7fe620005910 comp rx=0 tx=0).stop 2026-02-13T02:20:26.769 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.767+0000 7fe633cfd6c0 1 -- 10.20.193.27:0/3514685420 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe62c10eec0 msgr2=0x7fe62c0f83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.769 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.767+0000 7fe633cfd6c0 1 --2- 10.20.193.27:0/3514685420 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe62c10eec0 0x7fe62c0f83d0 secure :-1 s=READY pgs=124 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe61c005dc0 tx=0x7fe61c00bed0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.769 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.767+0000 7fe633cfd6c0 1 -- 10.20.193.27:0/3514685420 shutdown_connections 2026-02-13T02:20:26.769 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.767+0000 7fe633cfd6c0 1 --2- 10.20.193.27:0/3514685420 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe60803f950 0x7fe60805fe00 unknown :-1 s=CLOSED pgs=66 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.769 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.767+0000 7fe633cfd6c0 1 --2- 10.20.193.27:0/3514685420 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe62c1109f0 0x7fe62c0f8c10 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.769 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.767+0000 7fe633cfd6c0 1 --2- 10.20.193.27:0/3514685420 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe62c110040 0x7fe62c171360 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.769 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.767+0000 7fe633cfd6c0 1 --2- 10.20.193.27:0/3514685420 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe62c10eec0 0x7fe62c0f83d0 unknown :-1 s=CLOSED pgs=124 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.769 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.767+0000 7fe633cfd6c0 1 -- 10.20.193.27:0/3514685420 >> 10.20.193.27:0/3514685420 conn(0x7fe62c0ec5e0 msgr2=0x7fe62c0ee470 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.770 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.767+0000 7fe633cfd6c0 1 -- 10.20.193.27:0/3514685420 shutdown_connections 2026-02-13T02:20:26.770 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.767+0000 7fe633cfd6c0 1 -- 10.20.193.27:0/3514685420 wait complete. 2026-02-13T02:20:26.783 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.782+0000 7f21af7a96c0 1 -- 10.20.193.27:0/215433190 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f21a81726b0 con 0x7f21a810cc50 2026-02-13T02:20:26.786 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.784+0000 7f21967fc6c0 1 -- 10.20.193.27:0/215433190 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v29)=0 v29) ==== 121+0+0 (secure 0 0 0) 0x7f219800ae70 con 0x7f21a810cc50 2026-02-13T02:20:26.790 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.788+0000 7f21af7a96c0 1 -- 10.20.193.27:0/215433190 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f218403f8f0 msgr2=0x7f218405fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.790 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.788+0000 7f21af7a96c0 1 --2- 10.20.193.27:0/215433190 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f218403f8f0 0x7f218405fda0 secure :-1 s=READY pgs=67 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f219c02f940 tx=0x7f219c0023a0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.790 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.788+0000 7f21af7a96c0 1 -- 10.20.193.27:0/215433190 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f21a810cc50 msgr2=0x7f21a80f85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.790 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.788+0000 7f21af7a96c0 1 --2- 10.20.193.27:0/215433190 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f21a810cc50 0x7f21a80f85d0 secure :-1 s=READY pgs=126 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2198005dc0 tx=0x7f219800bbb0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.792 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.791+0000 7f21af7a96c0 1 -- 10.20.193.27:0/215433190 shutdown_connections 2026-02-13T02:20:26.792 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.791+0000 7f21af7a96c0 1 --2- 10.20.193.27:0/215433190 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f218403f8f0 0x7f218405fda0 unknown :-1 s=CLOSED pgs=67 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.792 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.791+0000 7f21af7a96c0 1 --2- 10.20.193.27:0/215433190 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f21a810e780 0x7f21a80f8e10 unknown :-1 s=CLOSED pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.792 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.791+0000 7f21af7a96c0 1 --2- 10.20.193.27:0/215433190 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f21a810ddd0 0x7f21a816f0f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.792 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.791+0000 7f21af7a96c0 1 --2- 10.20.193.27:0/215433190 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f21a810cc50 0x7f21a80f85d0 unknown :-1 s=CLOSED pgs=126 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.793 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.791+0000 7f21af7a96c0 1 -- 10.20.193.27:0/215433190 >> 10.20.193.27:0/215433190 conn(0x7f21a80ea370 msgr2=0x7f21a80616c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.795 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.793+0000 7f21af7a96c0 1 -- 10.20.193.27:0/215433190 shutdown_connections 2026-02-13T02:20:26.795 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.793+0000 7f21af7a96c0 1 -- 10.20.193.27:0/215433190 wait complete. 2026-02-13T02:20:26.798 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '707dbb29-0882-11f1-9262-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='trial027') 2026-02-13T02:20:26.800 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.798+0000 7f96d17fa6c0 1 -- 10.20.193.27:0/2203054161 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f96b80055c0 con 0x7f96ec10eed0 2026-02-13T02:20:26.801 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.800+0000 7fc879d096c0 1 -- 10.20.193.27:0/1391449949 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7fc874110550 con 0x7fc87410aa90 2026-02-13T02:20:26.802 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.801+0000 7f96d37fe6c0 1 -- 10.20.193.27:0/2203054161 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v30)=0 v30) ==== 125+0+0 (secure 0 0 0) 0x7f96dc035890 con 0x7f96ec10eed0 2026-02-13T02:20:26.803 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.802+0000 7fc870ff96c0 1 -- 10.20.193.27:0/1391449949 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v30)=0 v30) ==== 108+0+0 (secure 0 0 0) 0x7fc860002660 con 0x7fc87410aa90 2026-02-13T02:20:26.805 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.803+0000 7f96d17fa6c0 1 -- 10.20.193.27:0/2203054161 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f96c803f970 msgr2=0x7f96c805fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.805 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.803+0000 7f96d17fa6c0 1 --2- 10.20.193.27:0/2203054161 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f96c803f970 0x7f96c805fe20 secure :-1 s=READY pgs=68 gs=6 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f96e0004750 tx=0x7f96e0005ac0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.805 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.803+0000 7f96d17fa6c0 1 -- 10.20.193.27:0/2203054161 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f96ec10eed0 msgr2=0x7f96ec0f85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.803+0000 7f96d17fa6c0 1 --2- 10.20.193.27:0/2203054161 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f96ec10eed0 0x7f96ec0f85b0 secure :-1 s=READY pgs=128 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f96dc02f8e0 tx=0x7f96dc002960 comp rx=0 tx=0).stop 2026-02-13T02:20:26.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.803+0000 7f96d17fa6c0 1 -- 10.20.193.27:0/2203054161 shutdown_connections 2026-02-13T02:20:26.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.803+0000 7f96d17fa6c0 1 --2- 10.20.193.27:0/2203054161 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f96c803f970 0x7f96c805fe20 unknown :-1 s=CLOSED pgs=68 gs=6 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.803+0000 7f96d17fa6c0 1 --2- 10.20.193.27:0/2203054161 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f96ec110a00 0x7f96ec0f8e10 unknown :-1 s=CLOSED pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.803+0000 7f96d17fa6c0 1 --2- 10.20.193.27:0/2203054161 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f96ec110050 0x7f96ec171390 unknown :-1 s=CLOSED pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.803+0000 7f96d17fa6c0 1 --2- 10.20.193.27:0/2203054161 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f96ec10eed0 0x7f96ec0f85b0 unknown :-1 s=CLOSED pgs=128 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.803+0000 7f96d17fa6c0 1 -- 10.20.193.27:0/2203054161 >> 10.20.193.27:0/2203054161 conn(0x7f96ec0ec5f0 msgr2=0x7f96ec0eeb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.804+0000 7f96d17fa6c0 1 -- 10.20.193.27:0/2203054161 shutdown_connections 2026-02-13T02:20:26.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.804+0000 7f96d17fa6c0 1 -- 10.20.193.27:0/2203054161 wait complete. 2026-02-13T02:20:26.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.804+0000 7fc879d096c0 1 -- 10.20.193.27:0/1391449949 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc84803f940 msgr2=0x7fc84805fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.804+0000 7fc879d096c0 1 --2- 10.20.193.27:0/1391449949 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc84803f940 0x7fc84805fdf0 secure :-1 s=READY pgs=69 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc864005980 tx=0x7fc864005910 comp rx=0 tx=0).stop 2026-02-13T02:20:26.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.804+0000 7fc879d096c0 1 -- 10.20.193.27:0/1391449949 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc87410aa90 msgr2=0x7fc8740f2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.804+0000 7fc879d096c0 1 --2- 10.20.193.27:0/1391449949 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc87410aa90 0x7fc8740f2dc0 secure :-1 s=READY pgs=130 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc860005980 tx=0x7fc86000be00 comp rx=0 tx=0).stop 2026-02-13T02:20:26.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.804+0000 7fc879d096c0 1 -- 10.20.193.27:0/1391449949 shutdown_connections 2026-02-13T02:20:26.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.804+0000 7fc879d096c0 1 --2- 10.20.193.27:0/1391449949 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc84803f940 0x7fc84805fdf0 unknown :-1 s=CLOSED pgs=69 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.804+0000 7fc879d096c0 1 --2- 10.20.193.27:0/1391449949 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc87410c500 0x7fc8740f4000 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.804+0000 7fc879d096c0 1 --2- 10.20.193.27:0/1391449949 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc87410bb50 0x7fc8740f3410 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.804+0000 7fc879d096c0 1 --2- 10.20.193.27:0/1391449949 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc87410aa90 0x7fc8740f2dc0 unknown :-1 s=CLOSED pgs=130 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.804+0000 7fc879d096c0 1 -- 10.20.193.27:0/1391449949 >> 10.20.193.27:0/1391449949 conn(0x7fc8740e8510 msgr2=0x7fc874109900 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.804+0000 7fc879d096c0 1 -- 10.20.193.27:0/1391449949 shutdown_connections 2026-02-13T02:20:26.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.804+0000 7fc879d096c0 1 -- 10.20.193.27:0/1391449949 wait complete. 2026-02-13T02:20:26.812 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.811+0000 7f26bf5696c0 1 -- 10.20.193.27:0/2054756680 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f26b810f2a0 con 0x7f26b8110050 2026-02-13T02:20:26.815 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.813+0000 7f26a67fc6c0 1 -- 10.20.193.27:0/2054756680 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v31)=0 v31) ==== 121+0+0 (secure 0 0 0) 0x7f26ac005490 con 0x7f26b8110050 2026-02-13T02:20:26.819 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.817+0000 7f26bf5696c0 1 -- 10.20.193.27:0/2054756680 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f269403f9b0 msgr2=0x7f269405fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.817+0000 7f26bf5696c0 1 --2- 10.20.193.27:0/2054756680 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f269403f9b0 0x7f269405fe60 secure :-1 s=READY pgs=70 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f26a80029c0 tx=0x7f26a80023a0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.817+0000 7f26bf5696c0 1 -- 10.20.193.27:0/2054756680 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f26b8110050 msgr2=0x7f26b8113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.817+0000 7f26bf5696c0 1 --2- 10.20.193.27:0/2054756680 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f26b8110050 0x7f26b8113200 secure :-1 s=READY pgs=134 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f26ac005dc0 tx=0x7f26ac00bbd0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.818+0000 7f26bf5696c0 1 -- 10.20.193.27:0/2054756680 shutdown_connections 2026-02-13T02:20:26.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.818+0000 7f26bf5696c0 1 --2- 10.20.193.27:0/2054756680 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f269403f9b0 0x7f269405fe60 unknown :-1 s=CLOSED pgs=70 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.818+0000 7f26bf5696c0 1 --2- 10.20.193.27:0/2054756680 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f26b8110a00 0x7f26b8113df0 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.818+0000 7f26bf5696c0 1 --2- 10.20.193.27:0/2054756680 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f26b8110050 0x7f26b8113200 unknown :-1 s=CLOSED pgs=134 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.818+0000 7f26bf5696c0 1 --2- 10.20.193.27:0/2054756680 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f26b810eed0 0x7f26b8112bb0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.818+0000 7f26bf5696c0 1 -- 10.20.193.27:0/2054756680 >> 10.20.193.27:0/2054756680 conn(0x7f26b80ec5f0 msgr2=0x7f26b810c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.818+0000 7f26bf5696c0 1 -- 10.20.193.27:0/2054756680 shutdown_connections 2026-02-13T02:20:26.821 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.818+0000 7f26bf5696c0 1 -- 10.20.193.27:0/2054756680 wait complete. 2026-02-13T02:20:26.822 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.820+0000 7f60445b06c0 1 -- 10.20.193.27:0/2385354456 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f603c172790 con 0x7f603c10cc50 2026-02-13T02:20:26.825 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.824+0000 7f60237fe6c0 1 -- 10.20.193.27:0/2385354456 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v32)=0 v32) ==== 119+0+0 (secure 0 0 0) 0x7f602c00e8d0 con 0x7f603c10cc50 2026-02-13T02:20:26.829 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.827+0000 7f60445b06c0 1 -- 10.20.193.27:0/2385354456 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f601803f8f0 msgr2=0x7f601805fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.829 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.827+0000 7f60445b06c0 1 --2- 10.20.193.27:0/2385354456 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f601803f8f0 0x7f601805fda0 secure :-1 s=READY pgs=71 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f603002fa20 tx=0x7f6030005c00 comp rx=0 tx=0).stop 2026-02-13T02:20:26.829 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.827+0000 7f60445b06c0 1 -- 10.20.193.27:0/2385354456 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f603c10cc50 msgr2=0x7f603c061ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:26.829 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.827+0000 7f60445b06c0 1 --2- 10.20.193.27:0/2385354456 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f603c10cc50 0x7f603c061ea0 secure :-1 s=READY pgs=136 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f602c005dc0 tx=0x7f602c00bbd0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.829 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.827+0000 7f60445b06c0 1 -- 10.20.193.27:0/2385354456 shutdown_connections 2026-02-13T02:20:26.829 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.827+0000 7f60445b06c0 1 --2- 10.20.193.27:0/2385354456 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f601803f8f0 0x7f601805fda0 unknown :-1 s=CLOSED pgs=71 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.829 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.827+0000 7f60445b06c0 1 --2- 10.20.193.27:0/2385354456 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f603c10e780 0x7f603c0607a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.829 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.827+0000 7f60445b06c0 1 --2- 10.20.193.27:0/2385354456 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f603c10ddd0 0x7f603c16f0f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.829 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.827+0000 7f60445b06c0 1 --2- 10.20.193.27:0/2385354456 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f603c10cc50 0x7f603c061ea0 unknown :-1 s=CLOSED pgs=136 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:26.829 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.827+0000 7f60445b06c0 1 -- 10.20.193.27:0/2385354456 >> 10.20.193.27:0/2385354456 conn(0x7f603c0ea370 msgr2=0x7f603c0ec910 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:26.830 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.827+0000 7f60445b06c0 1 -- 10.20.193.27:0/2385354456 shutdown_connections 2026-02-13T02:20:26.830 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:26.827+0000 7f60445b06c0 1 -- 10.20.193.27:0/2385354456 wait complete. 2026-02-13T02:20:26.831 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '707dbb29-0882-11f1-9262-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='trial027') 2026-02-13T02:20:26.831 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '707dbb29-0882-11f1-9262-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='trial027') 2026-02-13T02:20:26.831 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '707dbb29-0882-11f1-9262-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='trial027') 2026-02-13T02:20:26.875 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '707dbb29-0882-11f1-9262-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='trial027') 2026-02-13T02:20:26.876 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '707dbb29-0882-11f1-9262-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='trial027') 2026-02-13T02:20:26.879 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '707dbb29-0882-11f1-9262-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='trial027') 2026-02-13T02:20:26.879 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '707dbb29-0882-11f1-9262-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='trial027') 2026-02-13T02:20:26.879 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '707dbb29-0882-11f1-9262-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='trial027') 2026-02-13T02:20:26.879 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '707dbb29-0882-11f1-9262-d404e6e7d460', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='trial027') 2026-02-13T02:20:26.880 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph config dump 2026-02-13T02:20:27.000 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:26 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:26.703+0000 7fdcb4638fc0 -1 mgr[py] Loading module stats in main interpreter 2026-02-13T02:20:27.000 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:26 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:26.747+0000 7fdcb4638fc0 -1 mgr[py] Loading module dashboard in main interpreter 2026-02-13T02:20:27.033 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:27.147 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.143+0000 7f44cdf676c0 1 -- 10.20.193.27:0/2238961805 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f44c8110040 msgr2=0x7f44c81104b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:27.147 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.143+0000 7f44cdf676c0 1 --2- 10.20.193.27:0/2238961805 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f44c8110040 0x7f44c81104b0 secure :-1 s=READY pgs=137 gs=1 cs=0 l=1 c_cookie=3556698537680704306 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f44b8009970 tx=0x7f44b802ed50 comp rx=0 tx=0).stop 2026-02-13T02:20:27.148 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.143+0000 7f44cdf676c0 1 -- 10.20.193.27:0/2238961805 shutdown_connections 2026-02-13T02:20:27.148 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.143+0000 7f44cdf676c0 1 --2- 10.20.193.27:0/2238961805 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f44c81109f0 0x7f44c8171380 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:27.148 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.143+0000 7f44cdf676c0 1 --2- 10.20.193.27:0/2238961805 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f44c8110040 0x7f44c81104b0 unknown :-1 s=CLOSED pgs=137 gs=1 cs=0 l=1 c_cookie=3556698537680704306 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:27.148 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.143+0000 7f44cdf676c0 1 --2- 10.20.193.27:0/2238961805 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f44c810eec0 0x7f44c810f2b0 unknown :-1 s=CLOSED pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:27.148 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.143+0000 7f44cdf676c0 1 -- 10.20.193.27:0/2238961805 >> 10.20.193.27:0/2238961805 conn(0x7f44c80ec5c0 msgr2=0x7f44c810c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:27.148 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.143+0000 7f44cdf676c0 1 -- 10.20.193.27:0/2238961805 shutdown_connections 2026-02-13T02:20:27.148 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.143+0000 7f44cdf676c0 1 -- 10.20.193.27:0/2238961805 wait complete. 2026-02-13T02:20:27.148 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.144+0000 7f44cdf676c0 1 Processor -- start 2026-02-13T02:20:27.148 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.144+0000 7f44cdf676c0 1 -- start start 2026-02-13T02:20:27.148 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.144+0000 7f44cdf676c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f44c810eec0 0x7f44c80f8670 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:27.149 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.144+0000 7f44cdf676c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f44c8110040 0x7f44c8171360 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:27.149 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.144+0000 7f44cdf676c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f44c81109f0 0x7f44c80f8eb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:27.149 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.144+0000 7f44cdf676c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f44c80f9a70 con 0x7f44c81109f0 2026-02-13T02:20:27.149 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.144+0000 7f44cdf676c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f44c80f9be0 con 0x7f44c810eec0 2026-02-13T02:20:27.149 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.144+0000 7f44cdf676c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f44c80f9da0 con 0x7f44c8110040 2026-02-13T02:20:27.149 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.144+0000 7f44c7fff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f44c81109f0 0x7f44c80f8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:27.149 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.144+0000 7f44c7fff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f44c81109f0 0x7f44c80f8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:59942/0 (socket says 10.20.193.27:59942) 2026-02-13T02:20:27.149 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.144+0000 7f44c7fff6c0 1 -- 10.20.193.27:0/978564177 learned_addr learned my addr 10.20.193.27:0/978564177 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:20:27.149 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.144+0000 7f44c77fe6c0 1 --2- 10.20.193.27:0/978564177 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f44c810eec0 0x7f44c80f8670 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:27.149 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.144+0000 7f44c6ffd6c0 1 --2- 10.20.193.27:0/978564177 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f44c8110040 0x7f44c8171360 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:27.149 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.144+0000 7f44c7fff6c0 1 -- 10.20.193.27:0/978564177 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f44c8110040 msgr2=0x7f44c8171360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:27.149 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.144+0000 7f44c7fff6c0 1 --2- 10.20.193.27:0/978564177 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f44c8110040 0x7f44c8171360 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:27.150 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.144+0000 7f44c7fff6c0 1 -- 10.20.193.27:0/978564177 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f44c810eec0 msgr2=0x7f44c80f8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:27.150 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.144+0000 7f44c7fff6c0 1 --2- 10.20.193.27:0/978564177 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f44c810eec0 0x7f44c80f8670 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:27.150 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.144+0000 7f44c7fff6c0 1 -- 10.20.193.27:0/978564177 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b8009590 con 0x7f44c81109f0 2026-02-13T02:20:27.150 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.144+0000 7f44c7fff6c0 1 --2- 10.20.193.27:0/978564177 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f44c81109f0 0x7f44c80f8eb0 secure :-1 s=READY pgs=138 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f44bc00c730 tx=0x7f44bc00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:27.150 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.144+0000 7f44c4ff96c0 1 -- 10.20.193.27:0/978564177 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f44bc003c40 con 0x7f44c81109f0 2026-02-13T02:20:27.150 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.144+0000 7f44cdf676c0 1 -- 10.20.193.27:0/978564177 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44c80fa020 con 0x7f44c81109f0 2026-02-13T02:20:27.150 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.144+0000 7f44c4ff96c0 1 -- 10.20.193.27:0/978564177 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f44bc00a070 con 0x7f44c81109f0 2026-02-13T02:20:27.150 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.144+0000 7f44c4ff96c0 1 -- 10.20.193.27:0/978564177 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f44bc00dca0 con 0x7f44c81109f0 2026-02-13T02:20:27.150 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.144+0000 7f44cdf676c0 1 -- 10.20.193.27:0/978564177 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44c80fa500 con 0x7f44c81109f0 2026-02-13T02:20:27.150 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.145+0000 7f44cdf676c0 1 -- 10.20.193.27:0/978564177 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44c8175260 con 0x7f44c81109f0 2026-02-13T02:20:27.150 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.145+0000 7f44c4ff96c0 1 -- 10.20.193.27:0/978564177 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 8) ==== 52814+0+0 (secure 0 0 0) 0x7f44bc005030 con 0x7f44c81109f0 2026-02-13T02:20:27.150 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.145+0000 7f44c4ff96c0 1 --2- 10.20.193.27:0/978564177 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f449c03f8f0 0x7f449c05fda0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:27.151 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.145+0000 7f44c4ff96c0 1 -- 10.20.193.27:0/978564177 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 935+0+0 (secure 0 0 0) 0x7f44bc052410 con 0x7f44c81109f0 2026-02-13T02:20:27.151 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.145+0000 7f44c77fe6c0 1 --2- 10.20.193.27:0/978564177 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f449c03f8f0 0x7f449c05fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:27.151 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.145+0000 7f44c77fe6c0 1 --2- 10.20.193.27:0/978564177 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f449c03f8f0 0x7f449c05fda0 secure :-1 s=READY pgs=72 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f44b4005980 tx=0x7f44b4005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:27.151 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.147+0000 7f44c4ff96c0 1 -- 10.20.193.27:0/978564177 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f44bc003da0 con 0x7f44c81109f0 2026-02-13T02:20:27.239 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.237+0000 7f44cdf676c0 1 -- 10.20.193.27:0/978564177 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f44c8174be0 con 0x7f44c81109f0 2026-02-13T02:20:27.239 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.238+0000 7f44c4ff96c0 1 -- 10.20.193.27:0/978564177 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v32) ==== 59+0+18512 (secure 0 0 0) 0x7f44c8174be0 con 0x7f44c81109f0 2026-02-13T02:20:27.241 INFO:teuthology.orchestra.run.trial027.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-13T02:20:27.241 INFO:teuthology.orchestra.run.trial027.stdout:global dev auth_debug true 2026-02-13T02:20:27.241 INFO:teuthology.orchestra.run.trial027.stdout:global basic container_image quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9 * 2026-02-13T02:20:27.241 INFO:teuthology.orchestra.run.trial027.stdout:global dev debug_asserts_on_shutdown true 2026-02-13T02:20:27.241 INFO:teuthology.orchestra.run.trial027.stdout:global basic log_to_file true 2026-02-13T02:20:27.241 INFO:teuthology.orchestra.run.trial027.stdout:global basic log_to_journald false 2026-02-13T02:20:27.241 INFO:teuthology.orchestra.run.trial027.stdout:global basic log_to_stderr false 2026-02-13T02:20:27.242 INFO:teuthology.orchestra.run.trial027.stdout:global advanced mon_allow_pool_delete true 2026-02-13T02:20:27.242 INFO:teuthology.orchestra.run.trial027.stdout:global advanced mon_clock_drift_allowed 1.000000 2026-02-13T02:20:27.242 INFO:teuthology.orchestra.run.trial027.stdout:global advanced mon_cluster_log_to_file true 2026-02-13T02:20:27.242 INFO:teuthology.orchestra.run.trial027.stdout:global advanced mon_max_pg_per_osd 10000 2026-02-13T02:20:27.242 INFO:teuthology.orchestra.run.trial027.stdout:global advanced mon_pg_warn_max_object_skew 0.000000 2026-02-13T02:20:27.242 INFO:teuthology.orchestra.run.trial027.stdout:global advanced mon_warn_on_crush_straw_calc_version_zero false 2026-02-13T02:20:27.242 INFO:teuthology.orchestra.run.trial027.stdout:global advanced mon_warn_on_legacy_crush_tunables false 2026-02-13T02:20:27.242 INFO:teuthology.orchestra.run.trial027.stdout:global advanced mon_warn_on_osd_down_out_interval_zero false 2026-02-13T02:20:27.242 INFO:teuthology.orchestra.run.trial027.stdout:global dev mon_warn_on_pool_pg_num_not_power_of_two false 2026-02-13T02:20:27.242 INFO:teuthology.orchestra.run.trial027.stdout:global advanced mon_warn_on_too_few_osds false 2026-02-13T02:20:27.242 INFO:teuthology.orchestra.run.trial027.stdout:global dev ms_die_on_bug true 2026-02-13T02:20:27.242 INFO:teuthology.orchestra.run.trial027.stdout:global dev ms_die_on_old_message true 2026-02-13T02:20:27.243 INFO:teuthology.orchestra.run.trial027.stdout:global advanced osd_pool_default_erasure_code_profile plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-13T02:20:27.243 INFO:teuthology.orchestra.run.trial027.stdout:global advanced osd_pool_default_pg_autoscale_mode off 2026-02-13T02:20:27.243 INFO:teuthology.orchestra.run.trial027.stdout:global advanced public_network 10.20.192.0/20 * 2026-02-13T02:20:27.243 INFO:teuthology.orchestra.run.trial027.stdout:mon advanced auth_allow_insecure_global_id_reclaim false 2026-02-13T02:20:27.243 INFO:teuthology.orchestra.run.trial027.stdout:mon advanced auth_mon_ticket_ttl 660.000000 2026-02-13T02:20:27.243 INFO:teuthology.orchestra.run.trial027.stdout:mon advanced auth_service_ticket_ttl 240.000000 2026-02-13T02:20:27.243 INFO:teuthology.orchestra.run.trial027.stdout:mon advanced debug_mon 20/20 2026-02-13T02:20:27.243 INFO:teuthology.orchestra.run.trial027.stdout:mon advanced debug_ms 1/1 2026-02-13T02:20:27.243 INFO:teuthology.orchestra.run.trial027.stdout:mon advanced debug_paxos 20/20 2026-02-13T02:20:27.243 INFO:teuthology.orchestra.run.trial027.stdout:mon advanced mon_data_avail_warn 5 2026-02-13T02:20:27.243 INFO:teuthology.orchestra.run.trial027.stdout:mon advanced mon_down_mkfs_grace 300 2026-02-13T02:20:27.244 INFO:teuthology.orchestra.run.trial027.stdout:mon advanced mon_mgr_mkfs_grace 240 2026-02-13T02:20:27.244 INFO:teuthology.orchestra.run.trial027.stdout:mon advanced mon_op_complaint_time 120 2026-02-13T02:20:27.244 INFO:teuthology.orchestra.run.trial027.stdout:mon dev mon_osd_prime_pg_temp true 2026-02-13T02:20:27.244 INFO:teuthology.orchestra.run.trial027.stdout:mon advanced mon_osd_reporter_subtree_level osd 2026-02-13T02:20:27.244 INFO:teuthology.orchestra.run.trial027.stdout:mon advanced mon_reweight_min_bytes_per_osd 10 2026-02-13T02:20:27.244 INFO:teuthology.orchestra.run.trial027.stdout:mon advanced mon_reweight_min_pgs_per_osd 4 2026-02-13T02:20:27.244 INFO:teuthology.orchestra.run.trial027.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim false 2026-02-13T02:20:27.244 INFO:teuthology.orchestra.run.trial027.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim_allowed false 2026-02-13T02:20:27.244 INFO:teuthology.orchestra.run.trial027.stdout:mgr advanced cephsqlite_lock_renewal_timeout 900000 2026-02-13T02:20:27.244 INFO:teuthology.orchestra.run.trial027.stdout:mgr advanced client_mount_timeout 30 2026-02-13T02:20:27.244 INFO:teuthology.orchestra.run.trial027.stdout:mgr advanced debug_client 20/20 2026-02-13T02:20:27.244 INFO:teuthology.orchestra.run.trial027.stdout:mgr advanced debug_mgr 20/20 2026-02-13T02:20:27.244 INFO:teuthology.orchestra.run.trial027.stdout:mgr advanced debug_ms 1/1 2026-02-13T02:20:27.245 INFO:teuthology.orchestra.run.trial027.stdout:mgr advanced mgr/cephadm/allow_ptrace true * 2026-02-13T02:20:27.245 INFO:teuthology.orchestra.run.trial027.stdout:mgr advanced mgr/cephadm/container_init True * 2026-02-13T02:20:27.245 INFO:teuthology.orchestra.run.trial027.stdout:mgr advanced mgr/cephadm/migration_current 8 * 2026-02-13T02:20:27.245 INFO:teuthology.orchestra.run.trial027.stdout:mgr advanced mgr/orchestrator/orchestrator cephadm 2026-02-13T02:20:27.245 INFO:teuthology.orchestra.run.trial027.stdout:mgr advanced mon_reweight_min_bytes_per_osd 10 2026-02-13T02:20:27.245 INFO:teuthology.orchestra.run.trial027.stdout:mgr advanced mon_reweight_min_pgs_per_osd 4 2026-02-13T02:20:27.245 INFO:teuthology.orchestra.run.trial027.stdout:mgr dev mon_warn_on_pool_no_app false 2026-02-13T02:20:27.245 INFO:teuthology.orchestra.run.trial027.stdout:osd dev bdev_debug_aio true 2026-02-13T02:20:27.245 INFO:teuthology.orchestra.run.trial027.stdout:osd dev bluestore_block_size 96636764160 * 2026-02-13T02:20:27.245 INFO:teuthology.orchestra.run.trial027.stdout:osd advanced bluestore_compression_mode aggressive 2026-02-13T02:20:27.245 INFO:teuthology.orchestra.run.trial027.stdout:osd dev bluestore_fsck_on_mount true 2026-02-13T02:20:27.246 INFO:teuthology.orchestra.run.trial027.stdout:osd advanced debug_bluefs 20/20 2026-02-13T02:20:27.246 INFO:teuthology.orchestra.run.trial027.stdout:osd advanced debug_bluestore 20/20 2026-02-13T02:20:27.246 INFO:teuthology.orchestra.run.trial027.stdout:osd advanced debug_ms 1/1 2026-02-13T02:20:27.246 INFO:teuthology.orchestra.run.trial027.stdout:osd advanced debug_osd 20/20 2026-02-13T02:20:27.246 INFO:teuthology.orchestra.run.trial027.stdout:osd advanced debug_rocksdb 10/10 2026-02-13T02:20:27.246 INFO:teuthology.orchestra.run.trial027.stdout:osd dev osd_debug_misdirected_ops true 2026-02-13T02:20:27.246 INFO:teuthology.orchestra.run.trial027.stdout:osd dev osd_debug_op_order true 2026-02-13T02:20:27.246 INFO:teuthology.orchestra.run.trial027.stdout:osd dev osd_debug_pg_log_writeout true 2026-02-13T02:20:27.246 INFO:teuthology.orchestra.run.trial027.stdout:osd dev osd_debug_shutdown true 2026-02-13T02:20:27.246 INFO:teuthology.orchestra.run.trial027.stdout:osd dev osd_debug_verify_cached_snaps true 2026-02-13T02:20:27.246 INFO:teuthology.orchestra.run.trial027.stdout:osd dev osd_debug_verify_missing_on_start true 2026-02-13T02:20:27.246 INFO:teuthology.orchestra.run.trial027.stdout:osd dev osd_debug_verify_stray_on_activate true 2026-02-13T02:20:27.246 INFO:teuthology.orchestra.run.trial027.stdout:osd advanced osd_deep_scrub_update_digest_min_age 30 2026-02-13T02:20:27.247 INFO:teuthology.orchestra.run.trial027.stdout:osd advanced osd_failsafe_full_ratio 0.950000 2026-02-13T02:20:27.247 INFO:teuthology.orchestra.run.trial027.stdout:osd advanced osd_max_omap_entries_per_request 10000 2026-02-13T02:20:27.247 INFO:teuthology.orchestra.run.trial027.stdout:osd advanced osd_mclock_profile high_recovery_ops 2026-02-13T02:20:27.247 INFO:teuthology.orchestra.run.trial027.stdout:osd dev osd_mclock_skip_benchmark true 2026-02-13T02:20:27.247 INFO:teuthology.orchestra.run.trial027.stdout:osd advanced osd_memory_target_autotune true 2026-02-13T02:20:27.247 INFO:teuthology.orchestra.run.trial027.stdout:osd advanced osd_objectstore bluestore * 2026-02-13T02:20:27.247 INFO:teuthology.orchestra.run.trial027.stdout:osd advanced osd_op_complaint_time 180.000000 2026-02-13T02:20:27.247 INFO:teuthology.orchestra.run.trial027.stdout:osd advanced osd_op_queue debug_random * 2026-02-13T02:20:27.247 INFO:teuthology.orchestra.run.trial027.stdout:osd advanced osd_op_queue_cut_off debug_random * 2026-02-13T02:20:27.247 INFO:teuthology.orchestra.run.trial027.stdout:osd advanced osd_recover_clone_overlap true 2026-02-13T02:20:27.247 INFO:teuthology.orchestra.run.trial027.stdout:osd advanced osd_recovery_max_chunk 1048576 2026-02-13T02:20:27.247 INFO:teuthology.orchestra.run.trial027.stdout:osd advanced osd_scrub_load_threshold 5.000000 2026-02-13T02:20:27.247 INFO:teuthology.orchestra.run.trial027.stdout:osd advanced osd_scrub_max_interval 600.000000 2026-02-13T02:20:27.248 INFO:teuthology.orchestra.run.trial027.stdout:osd advanced osd_shutdown_pgref_assert true 2026-02-13T02:20:27.248 INFO:teuthology.orchestra.run.trial027.stdout:mds advanced debug_mds 20/20 2026-02-13T02:20:27.248 INFO:teuthology.orchestra.run.trial027.stdout:mds advanced debug_mds_balancer 20/20 2026-02-13T02:20:27.248 INFO:teuthology.orchestra.run.trial027.stdout:mds advanced debug_ms 2/2 2026-02-13T02:20:27.248 INFO:teuthology.orchestra.run.trial027.stdout:mds advanced mds_bal_fragment_size_max 10000 2026-02-13T02:20:27.248 INFO:teuthology.orchestra.run.trial027.stdout:mds advanced mds_bal_merge_size 5 2026-02-13T02:20:27.248 INFO:teuthology.orchestra.run.trial027.stdout:mds advanced mds_bal_split_bits 3 2026-02-13T02:20:27.248 INFO:teuthology.orchestra.run.trial027.stdout:mds advanced mds_bal_split_size 100 2026-02-13T02:20:27.248 INFO:teuthology.orchestra.run.trial027.stdout:mds dev mds_debug_frag true 2026-02-13T02:20:27.248 INFO:teuthology.orchestra.run.trial027.stdout:mds dev mds_debug_scatterstat true 2026-02-13T02:20:27.248 INFO:teuthology.orchestra.run.trial027.stdout:mds advanced mds_op_complaint_time 180.000000 2026-02-13T02:20:27.248 INFO:teuthology.orchestra.run.trial027.stdout:mds dev mds_verify_scatter true 2026-02-13T02:20:27.248 INFO:teuthology.orchestra.run.trial027.stdout:mds advanced osd_op_complaint_time 180.000000 2026-02-13T02:20:27.249 INFO:teuthology.orchestra.run.trial027.stdout:mds advanced rados_mon_op_timeout 900 2026-02-13T02:20:27.249 INFO:teuthology.orchestra.run.trial027.stdout:mds advanced rados_osd_op_timeout 900 2026-02-13T02:20:27.249 INFO:teuthology.orchestra.run.trial027.stdout:client dev client_fscrypt_dummy_encryption true 2026-02-13T02:20:27.249 INFO:teuthology.orchestra.run.trial027.stdout:client advanced client_mount_timeout 600 2026-02-13T02:20:27.249 INFO:teuthology.orchestra.run.trial027.stdout:client advanced debug_client 20/20 2026-02-13T02:20:27.249 INFO:teuthology.orchestra.run.trial027.stdout:client advanced debug_ms 1/1 2026-02-13T02:20:27.249 INFO:teuthology.orchestra.run.trial027.stdout:client advanced rados_mon_op_timeout 900 2026-02-13T02:20:27.249 INFO:teuthology.orchestra.run.trial027.stdout:client advanced rados_osd_op_timeout 900 2026-02-13T02:20:27.249 INFO:teuthology.orchestra.run.trial027.stdout:client.rgw advanced rgw_cache_enabled true 2026-02-13T02:20:27.249 INFO:teuthology.orchestra.run.trial027.stdout:client.rgw advanced rgw_enable_ops_log true 2026-02-13T02:20:27.249 INFO:teuthology.orchestra.run.trial027.stdout:client.rgw advanced rgw_enable_usage_log true 2026-02-13T02:20:27.250 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.239+0000 7f44cdf676c0 1 -- 10.20.193.27:0/978564177 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f449c03f8f0 msgr2=0x7f449c05fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:27.250 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.239+0000 7f44cdf676c0 1 --2- 10.20.193.27:0/978564177 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f449c03f8f0 0x7f449c05fda0 secure :-1 s=READY pgs=72 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f44b4005980 tx=0x7f44b4005910 comp rx=0 tx=0).stop 2026-02-13T02:20:27.251 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.239+0000 7f44cdf676c0 1 -- 10.20.193.27:0/978564177 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f44c81109f0 msgr2=0x7f44c80f8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:27.251 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.239+0000 7f44cdf676c0 1 --2- 10.20.193.27:0/978564177 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f44c81109f0 0x7f44c80f8eb0 secure :-1 s=READY pgs=138 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f44bc00c730 tx=0x7f44bc00cc00 comp rx=0 tx=0).stop 2026-02-13T02:20:27.251 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.239+0000 7f44cdf676c0 1 -- 10.20.193.27:0/978564177 shutdown_connections 2026-02-13T02:20:27.251 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.239+0000 7f44cdf676c0 1 --2- 10.20.193.27:0/978564177 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f449c03f8f0 0x7f449c05fda0 unknown :-1 s=CLOSED pgs=72 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:27.251 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.239+0000 7f44cdf676c0 1 --2- 10.20.193.27:0/978564177 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f44c81109f0 0x7f44c80f8eb0 unknown :-1 s=CLOSED pgs=138 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:27.251 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.239+0000 7f44cdf676c0 1 --2- 10.20.193.27:0/978564177 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f44c8110040 0x7f44c8171360 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:27.251 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.239+0000 7f44cdf676c0 1 --2- 10.20.193.27:0/978564177 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f44c810eec0 0x7f44c80f8670 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:27.251 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.239+0000 7f44cdf676c0 1 -- 10.20.193.27:0/978564177 >> 10.20.193.27:0/978564177 conn(0x7f44c80ec5c0 msgr2=0x7f44c80eec20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:27.251 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.239+0000 7f44cdf676c0 1 -- 10.20.193.27:0/978564177 shutdown_connections 2026-02-13T02:20:27.251 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:27.239+0000 7f44cdf676c0 1 -- 10.20.193.27:0/978564177 wait complete. 2026-02-13T02:20:27.279 INFO:tasks.cephadm:Deploying OSDs... 2026-02-13T02:20:27.280 DEBUG:teuthology.orchestra.run.trial027:> set -ex 2026-02-13T02:20:27.280 DEBUG:teuthology.orchestra.run.trial027:> dd if=/scratch_devs of=/dev/stdout 2026-02-13T02:20:27.298 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-13T02:20:27.298 DEBUG:teuthology.orchestra.run.trial027:> stat /dev/vg_nvme/lv_1 2026-02-13T02:20:27.305 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;trial027=x;trial028=y;trial107=z", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:20:27.306 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1964660631' entity='client.admin' 2026-02-13T02:20:27.306 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2795753423' entity='client.admin' 2026-02-13T02:20:27.306 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/4160888720' entity='client.admin' 2026-02-13T02:20:27.306 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3436891875' entity='client.admin' 2026-02-13T02:20:27.306 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:27.306 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1595880533' entity='client.admin' 2026-02-13T02:20:27.306 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/318736707' entity='client.admin' 2026-02-13T02:20:27.306 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1874777830' entity='client.admin' 2026-02-13T02:20:27.306 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2920323902' entity='client.admin' 2026-02-13T02:20:27.306 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2673424342' entity='client.admin' 2026-02-13T02:20:27.306 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2718718715' entity='client.admin' 2026-02-13T02:20:27.307 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/4073212428' entity='client.admin' 2026-02-13T02:20:27.307 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2232626932' entity='client.admin' 2026-02-13T02:20:27.307 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/416812117' entity='client.admin' 2026-02-13T02:20:27.307 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:27.307 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1855195464' entity='client.admin' 2026-02-13T02:20:27.307 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:27.307 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:27.307 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:27.307 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2026-02-13T02:20:27.307 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2026-02-13T02:20:27.307 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2026-02-13T02:20:27.307 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:27.307 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1094763007' entity='client.admin' 2026-02-13T02:20:27.308 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/357666803' entity='client.admin' 2026-02-13T02:20:27.308 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/4254534103' entity='client.admin' 2026-02-13T02:20:27.308 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3514685420' entity='client.admin' 2026-02-13T02:20:27.308 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/215433190' entity='client.admin' 2026-02-13T02:20:27.308 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2203054161' entity='client.admin' 2026-02-13T02:20:27.308 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2054756680' entity='client.admin' 2026-02-13T02:20:27.308 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:27 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2385354456' entity='client.admin' 2026-02-13T02:20:27.308 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:27.004+0000 7fdcb4638fc0 -1 mgr[py] Loading module mirroring in main interpreter 2026-02-13T02:20:27.308 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:27.013+0000 7fdcb4638fc0 -1 mgr[py] Loading module snap_schedule in main interpreter 2026-02-13T02:20:27.309 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:27.020+0000 7fdcb4638fc0 -1 mgr[py] Loading module prometheus in main interpreter 2026-02-13T02:20:27.309 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:27.026+0000 7fdcb4638fc0 -1 mgr[py] Loading module balancer in main interpreter 2026-02-13T02:20:27.309 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:27.032+0000 7fdcb4638fc0 -1 mgr[py] Loading module rbd_support in main interpreter 2026-02-13T02:20:27.309 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:27.044+0000 7fdcb4638fc0 -1 mgr[py] Loading module smb in main interpreter 2026-02-13T02:20:27.309 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:27.069+0000 7fdcb4638fc0 -1 mgr[py] Loading module devicehealth in main interpreter 2026-02-13T02:20:27.309 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:27.072+0000 7fdcb4638fc0 -1 mgr[py] Loading module alerts in main interpreter 2026-02-13T02:20:27.309 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:27.073+0000 7fdcb4638fc0 -1 mgr[py] Loading module telemetry in main interpreter 2026-02-13T02:20:27.309 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:27.080+0000 7fdcb4638fc0 -1 mgr[py] Loading module diskprediction_local in main interpreter 2026-02-13T02:20:27.309 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:27.117+0000 7fdcb4638fc0 -1 mgr[py] Loading module status in main interpreter 2026-02-13T02:20:27.310 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:27.119+0000 7fdcb4638fc0 -1 mgr[py] Loading module rgw in main interpreter 2026-02-13T02:20:27.310 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:27.123+0000 7fdcb4638fc0 -1 mgr[py] Loading module insights in main interpreter 2026-02-13T02:20:27.310 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:27.125+0000 7fdcb4638fc0 -1 mgr[py] Loading module iostat in main interpreter 2026-02-13T02:20:27.310 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:27.125+0000 7fdcb4638fc0 -1 mgr[py] Loading module pg_autoscaler in main interpreter 2026-02-13T02:20:27.310 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:27.128+0000 7fdcb4638fc0 -1 mgr[py] Loading module osd_perf_query in main interpreter 2026-02-13T02:20:27.310 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:27.129+0000 7fdcb4638fc0 -1 mgr[py] Loading module crash in main interpreter 2026-02-13T02:20:27.310 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:27.132+0000 7fdcb4638fc0 -1 mgr[py] Loading module progress in main interpreter 2026-02-13T02:20:27.311 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:27.135+0000 7fdcb4638fc0 -1 mgr[py] Loading module rook in main interpreter 2026-02-13T02:20:27.311 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:27.307+0000 7fdcb4638fc0 -1 mgr[py] Loading module nfs in main interpreter 2026-02-13T02:20:27.311 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:27.308+0000 7fdcb4638fc0 -1 mgr[py] Loading module telegraf in main interpreter 2026-02-13T02:20:27.354 INFO:teuthology.orchestra.run.trial027.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-13T02:20:27.354 INFO:teuthology.orchestra.run.trial027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-13T02:20:27.354 INFO:teuthology.orchestra.run.trial027.stdout:Device: 6h/6d Inode: 1297 Links: 1 2026-02-13T02:20:27.355 INFO:teuthology.orchestra.run.trial027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-13T02:20:27.355 INFO:teuthology.orchestra.run.trial027.stdout:Context: system_u:object_r:device_t:s0 2026-02-13T02:20:27.355 INFO:teuthology.orchestra.run.trial027.stdout:Access: 2026-02-13 02:20:04.131148058 +0000 2026-02-13T02:20:27.355 INFO:teuthology.orchestra.run.trial027.stdout:Modify: 2026-02-13 02:18:20.242507206 +0000 2026-02-13T02:20:27.355 INFO:teuthology.orchestra.run.trial027.stdout:Change: 2026-02-13 02:18:20.242507206 +0000 2026-02-13T02:20:27.355 INFO:teuthology.orchestra.run.trial027.stdout: Birth: 2026-02-13 02:18:20.242507206 +0000 2026-02-13T02:20:27.355 DEBUG:teuthology.orchestra.run.trial027:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-13T02:20:27.384 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;trial027=x;trial028=y;trial107=z", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:20:27.384 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1964660631' entity='client.admin' 2026-02-13T02:20:27.384 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2795753423' entity='client.admin' 2026-02-13T02:20:27.384 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/4160888720' entity='client.admin' 2026-02-13T02:20:27.384 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3436891875' entity='client.admin' 2026-02-13T02:20:27.384 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:27.385 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1595880533' entity='client.admin' 2026-02-13T02:20:27.385 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/318736707' entity='client.admin' 2026-02-13T02:20:27.385 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1874777830' entity='client.admin' 2026-02-13T02:20:27.385 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2920323902' entity='client.admin' 2026-02-13T02:20:27.385 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2673424342' entity='client.admin' 2026-02-13T02:20:27.385 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2718718715' entity='client.admin' 2026-02-13T02:20:27.385 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/4073212428' entity='client.admin' 2026-02-13T02:20:27.385 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2232626932' entity='client.admin' 2026-02-13T02:20:27.385 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/416812117' entity='client.admin' 2026-02-13T02:20:27.385 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:27.385 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1855195464' entity='client.admin' 2026-02-13T02:20:27.385 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:27.385 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:27.386 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:27.386 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2026-02-13T02:20:27.386 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2026-02-13T02:20:27.386 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2026-02-13T02:20:27.386 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:27.386 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1094763007' entity='client.admin' 2026-02-13T02:20:27.386 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/357666803' entity='client.admin' 2026-02-13T02:20:27.386 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/4254534103' entity='client.admin' 2026-02-13T02:20:27.386 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3514685420' entity='client.admin' 2026-02-13T02:20:27.386 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/215433190' entity='client.admin' 2026-02-13T02:20:27.386 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2203054161' entity='client.admin' 2026-02-13T02:20:27.386 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2054756680' entity='client.admin' 2026-02-13T02:20:27.387 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:27 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2385354456' entity='client.admin' 2026-02-13T02:20:27.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;trial027=x;trial028=y;trial107=z", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:20:27.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1964660631' entity='client.admin' 2026-02-13T02:20:27.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2795753423' entity='client.admin' 2026-02-13T02:20:27.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/4160888720' entity='client.admin' 2026-02-13T02:20:27.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3436891875' entity='client.admin' 2026-02-13T02:20:27.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:27.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1595880533' entity='client.admin' 2026-02-13T02:20:27.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/318736707' entity='client.admin' 2026-02-13T02:20:27.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1874777830' entity='client.admin' 2026-02-13T02:20:27.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2920323902' entity='client.admin' 2026-02-13T02:20:27.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2673424342' entity='client.admin' 2026-02-13T02:20:27.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2718718715' entity='client.admin' 2026-02-13T02:20:27.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/4073212428' entity='client.admin' 2026-02-13T02:20:27.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2232626932' entity='client.admin' 2026-02-13T02:20:27.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/416812117' entity='client.admin' 2026-02-13T02:20:27.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:27.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1855195464' entity='client.admin' 2026-02-13T02:20:27.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:27.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:27.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:27.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2026-02-13T02:20:27.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2026-02-13T02:20:27.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2026-02-13T02:20:27.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:27.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1094763007' entity='client.admin' 2026-02-13T02:20:27.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/357666803' entity='client.admin' 2026-02-13T02:20:27.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/4254534103' entity='client.admin' 2026-02-13T02:20:27.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3514685420' entity='client.admin' 2026-02-13T02:20:27.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/215433190' entity='client.admin' 2026-02-13T02:20:27.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2203054161' entity='client.admin' 2026-02-13T02:20:27.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2054756680' entity='client.admin' 2026-02-13T02:20:27.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:27 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2385354456' entity='client.admin' 2026-02-13T02:20:27.420 INFO:teuthology.orchestra.run.trial027.stderr:1+0 records in 2026-02-13T02:20:27.420 INFO:teuthology.orchestra.run.trial027.stderr:1+0 records out 2026-02-13T02:20:27.420 INFO:teuthology.orchestra.run.trial027.stderr:512 bytes copied, 0.000164467 s, 3.1 MB/s 2026-02-13T02:20:27.422 DEBUG:teuthology.orchestra.run.trial027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-13T02:20:27.480 DEBUG:teuthology.orchestra.run.trial027:> stat /dev/vg_nvme/lv_2 2026-02-13T02:20:27.536 INFO:teuthology.orchestra.run.trial027.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-13T02:20:27.537 INFO:teuthology.orchestra.run.trial027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-13T02:20:27.537 INFO:teuthology.orchestra.run.trial027.stdout:Device: 6h/6d Inode: 1301 Links: 1 2026-02-13T02:20:27.537 INFO:teuthology.orchestra.run.trial027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-13T02:20:27.537 INFO:teuthology.orchestra.run.trial027.stdout:Context: system_u:object_r:device_t:s0 2026-02-13T02:20:27.537 INFO:teuthology.orchestra.run.trial027.stdout:Access: 2026-02-13 02:20:04.131148058 +0000 2026-02-13T02:20:27.537 INFO:teuthology.orchestra.run.trial027.stdout:Modify: 2026-02-13 02:18:20.242507206 +0000 2026-02-13T02:20:27.537 INFO:teuthology.orchestra.run.trial027.stdout:Change: 2026-02-13 02:18:20.242507206 +0000 2026-02-13T02:20:27.537 INFO:teuthology.orchestra.run.trial027.stdout: Birth: 2026-02-13 02:18:20.242507206 +0000 2026-02-13T02:20:27.538 DEBUG:teuthology.orchestra.run.trial027:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-13T02:20:27.599 INFO:teuthology.orchestra.run.trial027.stderr:1+0 records in 2026-02-13T02:20:27.600 INFO:teuthology.orchestra.run.trial027.stderr:1+0 records out 2026-02-13T02:20:27.600 INFO:teuthology.orchestra.run.trial027.stderr:512 bytes copied, 0.000140973 s, 3.6 MB/s 2026-02-13T02:20:27.601 DEBUG:teuthology.orchestra.run.trial027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-13T02:20:27.659 DEBUG:teuthology.orchestra.run.trial027:> stat /dev/vg_nvme/lv_3 2026-02-13T02:20:27.714 INFO:teuthology.orchestra.run.trial027.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-13T02:20:27.714 INFO:teuthology.orchestra.run.trial027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-13T02:20:27.715 INFO:teuthology.orchestra.run.trial027.stdout:Device: 6h/6d Inode: 1310 Links: 1 2026-02-13T02:20:27.715 INFO:teuthology.orchestra.run.trial027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-13T02:20:27.715 INFO:teuthology.orchestra.run.trial027.stdout:Context: system_u:object_r:device_t:s0 2026-02-13T02:20:27.715 INFO:teuthology.orchestra.run.trial027.stdout:Access: 2026-02-13 02:20:04.131148058 +0000 2026-02-13T02:20:27.715 INFO:teuthology.orchestra.run.trial027.stdout:Modify: 2026-02-13 02:18:20.242507206 +0000 2026-02-13T02:20:27.715 INFO:teuthology.orchestra.run.trial027.stdout:Change: 2026-02-13 02:18:20.242507206 +0000 2026-02-13T02:20:27.715 INFO:teuthology.orchestra.run.trial027.stdout: Birth: 2026-02-13 02:18:20.242507206 +0000 2026-02-13T02:20:27.716 DEBUG:teuthology.orchestra.run.trial027:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-13T02:20:27.743 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:27 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:27.559+0000 7f6f54b64fc0 -1 mgr[py] Loading module dashboard in main interpreter 2026-02-13T02:20:27.750 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:27.310+0000 7fdcb4638fc0 -1 mgr[py] Loading module selftest in main interpreter 2026-02-13T02:20:27.750 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:27.313+0000 7fdcb4638fc0 -1 mgr[py] Loading module test_orchestrator in main interpreter 2026-02-13T02:20:27.750 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:27.314+0000 7fdcb4638fc0 -1 mgr[py] Loading module osd_support in main interpreter 2026-02-13T02:20:27.750 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:27.315+0000 7fdcb4638fc0 -1 mgr[py] Loading module orchestrator in main interpreter 2026-02-13T02:20:27.750 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:27.315+0000 7fdcb4638fc0 -1 mgr[py] Loading module cephadm in main interpreter 2026-02-13T02:20:27.751 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:27.457+0000 7fdcb4638fc0 -1 mgr[py] Loading module volumes in main interpreter 2026-02-13T02:20:27.751 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:27.486+0000 7fdcb4638fc0 -1 mgr[py] Loading module k8sevents in main interpreter 2026-02-13T02:20:27.751 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:27.490+0000 7fdcb4638fc0 -1 mgr[py] Loading module mds_autoscaler in main interpreter 2026-02-13T02:20:27.751 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:27.491+0000 7fdcb4638fc0 -1 mgr[py] Loading module localpool in main interpreter 2026-02-13T02:20:27.751 INFO:journalctl@ceph.mgr.z.trial107.stdout:Feb 13 02:20:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-z[28758]: 2026-02-13T02:20:27.492+0000 7fdcb4638fc0 -1 mgr[py] Loading module influx in main interpreter 2026-02-13T02:20:27.777 INFO:teuthology.orchestra.run.trial027.stderr:1+0 records in 2026-02-13T02:20:27.778 INFO:teuthology.orchestra.run.trial027.stderr:1+0 records out 2026-02-13T02:20:27.778 INFO:teuthology.orchestra.run.trial027.stderr:512 bytes copied, 0.000170388 s, 3.0 MB/s 2026-02-13T02:20:27.779 DEBUG:teuthology.orchestra.run.trial027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-13T02:20:27.837 DEBUG:teuthology.orchestra.run.trial027:> stat /dev/vg_nvme/lv_4 2026-02-13T02:20:27.893 INFO:teuthology.orchestra.run.trial027.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-13T02:20:27.893 INFO:teuthology.orchestra.run.trial027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-13T02:20:27.893 INFO:teuthology.orchestra.run.trial027.stdout:Device: 6h/6d Inode: 1312 Links: 1 2026-02-13T02:20:27.893 INFO:teuthology.orchestra.run.trial027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-13T02:20:27.894 INFO:teuthology.orchestra.run.trial027.stdout:Context: system_u:object_r:device_t:s0 2026-02-13T02:20:27.894 INFO:teuthology.orchestra.run.trial027.stdout:Access: 2026-02-13 02:20:04.131148058 +0000 2026-02-13T02:20:27.894 INFO:teuthology.orchestra.run.trial027.stdout:Modify: 2026-02-13 02:18:20.242507206 +0000 2026-02-13T02:20:27.894 INFO:teuthology.orchestra.run.trial027.stdout:Change: 2026-02-13 02:18:20.242507206 +0000 2026-02-13T02:20:27.894 INFO:teuthology.orchestra.run.trial027.stdout: Birth: 2026-02-13 02:18:20.242507206 +0000 2026-02-13T02:20:27.894 DEBUG:teuthology.orchestra.run.trial027:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-13T02:20:27.955 INFO:teuthology.orchestra.run.trial027.stderr:1+0 records in 2026-02-13T02:20:27.956 INFO:teuthology.orchestra.run.trial027.stderr:1+0 records out 2026-02-13T02:20:27.956 INFO:teuthology.orchestra.run.trial027.stderr:512 bytes copied, 0.000167704 s, 3.1 MB/s 2026-02-13T02:20:27.957 DEBUG:teuthology.orchestra.run.trial027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-13T02:20:28.015 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-13T02:20:28.015 DEBUG:teuthology.orchestra.run.trial028:> dd if=/scratch_devs of=/dev/stdout 2026-02-13T02:20:28.030 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-13T02:20:28.031 DEBUG:teuthology.orchestra.run.trial028:> stat /dev/vg_nvme/lv_1 2026-02-13T02:20:28.088 INFO:teuthology.orchestra.run.trial028.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-13T02:20:28.088 INFO:teuthology.orchestra.run.trial028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-13T02:20:28.088 INFO:teuthology.orchestra.run.trial028.stdout:Device: 6h/6d Inode: 1298 Links: 1 2026-02-13T02:20:28.088 INFO:teuthology.orchestra.run.trial028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-13T02:20:28.088 INFO:teuthology.orchestra.run.trial028.stdout:Context: system_u:object_r:device_t:s0 2026-02-13T02:20:28.088 INFO:teuthology.orchestra.run.trial028.stdout:Access: 2026-02-13 02:20:27.340969632 +0000 2026-02-13T02:20:28.088 INFO:teuthology.orchestra.run.trial028.stdout:Modify: 2026-02-13 02:18:20.119110203 +0000 2026-02-13T02:20:28.088 INFO:teuthology.orchestra.run.trial028.stdout:Change: 2026-02-13 02:18:20.119110203 +0000 2026-02-13T02:20:28.088 INFO:teuthology.orchestra.run.trial028.stdout: Birth: 2026-02-13 02:18:20.119110203 +0000 2026-02-13T02:20:28.089 DEBUG:teuthology.orchestra.run.trial028:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-13T02:20:28.131 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:27 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:27.814+0000 7f6f54b64fc0 -1 mgr[py] Loading module mirroring in main interpreter 2026-02-13T02:20:28.131 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:27 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:27.823+0000 7f6f54b64fc0 -1 mgr[py] Loading module snap_schedule in main interpreter 2026-02-13T02:20:28.132 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:27 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:27.829+0000 7f6f54b64fc0 -1 mgr[py] Loading module prometheus in main interpreter 2026-02-13T02:20:28.132 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:27 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:27.836+0000 7f6f54b64fc0 -1 mgr[py] Loading module balancer in main interpreter 2026-02-13T02:20:28.132 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:27 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:27.842+0000 7f6f54b64fc0 -1 mgr[py] Loading module rbd_support in main interpreter 2026-02-13T02:20:28.132 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:27 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:27.854+0000 7f6f54b64fc0 -1 mgr[py] Loading module smb in main interpreter 2026-02-13T02:20:28.132 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:27 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:27.879+0000 7f6f54b64fc0 -1 mgr[py] Loading module devicehealth in main interpreter 2026-02-13T02:20:28.132 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:27 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:27.882+0000 7f6f54b64fc0 -1 mgr[py] Loading module alerts in main interpreter 2026-02-13T02:20:28.132 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:27 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:27.884+0000 7f6f54b64fc0 -1 mgr[py] Loading module telemetry in main interpreter 2026-02-13T02:20:28.132 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:27 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:27.891+0000 7f6f54b64fc0 -1 mgr[py] Loading module diskprediction_local in main interpreter 2026-02-13T02:20:28.132 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:27 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:27.929+0000 7f6f54b64fc0 -1 mgr[py] Loading module status in main interpreter 2026-02-13T02:20:28.132 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:27 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:27.931+0000 7f6f54b64fc0 -1 mgr[py] Loading module rgw in main interpreter 2026-02-13T02:20:28.132 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:27 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:27.935+0000 7f6f54b64fc0 -1 mgr[py] Loading module insights in main interpreter 2026-02-13T02:20:28.133 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:27 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:27.937+0000 7f6f54b64fc0 -1 mgr[py] Loading module iostat in main interpreter 2026-02-13T02:20:28.133 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:27 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:27.938+0000 7f6f54b64fc0 -1 mgr[py] Loading module pg_autoscaler in main interpreter 2026-02-13T02:20:28.133 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:27 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:27.941+0000 7f6f54b64fc0 -1 mgr[py] Loading module osd_perf_query in main interpreter 2026-02-13T02:20:28.133 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:27 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:27.942+0000 7f6f54b64fc0 -1 mgr[py] Loading module crash in main interpreter 2026-02-13T02:20:28.133 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:27 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:27.944+0000 7f6f54b64fc0 -1 mgr[py] Loading module progress in main interpreter 2026-02-13T02:20:28.133 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:27 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:27.947+0000 7f6f54b64fc0 -1 mgr[py] Loading module rook in main interpreter 2026-02-13T02:20:28.133 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:28 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:28.131+0000 7f6f54b64fc0 -1 mgr[py] Loading module nfs in main interpreter 2026-02-13T02:20:28.133 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:28 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:28.131+0000 7f6f54b64fc0 -1 mgr[py] Loading module telegraf in main interpreter 2026-02-13T02:20:28.155 INFO:teuthology.orchestra.run.trial028.stderr:1+0 records in 2026-02-13T02:20:28.156 INFO:teuthology.orchestra.run.trial028.stderr:1+0 records out 2026-02-13T02:20:28.156 INFO:teuthology.orchestra.run.trial028.stderr:512 bytes copied, 0.000181187 s, 2.8 MB/s 2026-02-13T02:20:28.157 DEBUG:teuthology.orchestra.run.trial028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-13T02:20:28.214 DEBUG:teuthology.orchestra.run.trial028:> stat /dev/vg_nvme/lv_2 2026-02-13T02:20:28.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:28 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/978564177' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2026-02-13T02:20:28.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:28 trial107 ceph-mon[27422]: Standby manager daemon z started 2026-02-13T02:20:28.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:28 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:28.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:28 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:28.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:28 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:28.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:28 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:28.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:28 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:28.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:28 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:28.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:28 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:28.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:28 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:28.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:28 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:28.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:28 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:28.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:28 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:28.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:28 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:28.270 INFO:teuthology.orchestra.run.trial028.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-13T02:20:28.270 INFO:teuthology.orchestra.run.trial028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-13T02:20:28.270 INFO:teuthology.orchestra.run.trial028.stdout:Device: 6h/6d Inode: 1299 Links: 1 2026-02-13T02:20:28.271 INFO:teuthology.orchestra.run.trial028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-13T02:20:28.271 INFO:teuthology.orchestra.run.trial028.stdout:Context: system_u:object_r:device_t:s0 2026-02-13T02:20:28.271 INFO:teuthology.orchestra.run.trial028.stdout:Access: 2026-02-13 02:20:27.340969632 +0000 2026-02-13T02:20:28.271 INFO:teuthology.orchestra.run.trial028.stdout:Modify: 2026-02-13 02:18:20.119110203 +0000 2026-02-13T02:20:28.271 INFO:teuthology.orchestra.run.trial028.stdout:Change: 2026-02-13 02:18:20.119110203 +0000 2026-02-13T02:20:28.271 INFO:teuthology.orchestra.run.trial028.stdout: Birth: 2026-02-13 02:18:20.119110203 +0000 2026-02-13T02:20:28.271 DEBUG:teuthology.orchestra.run.trial028:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-13T02:20:28.334 INFO:teuthology.orchestra.run.trial028.stderr:1+0 records in 2026-02-13T02:20:28.334 INFO:teuthology.orchestra.run.trial028.stderr:1+0 records out 2026-02-13T02:20:28.334 INFO:teuthology.orchestra.run.trial028.stderr:512 bytes copied, 0.00014503 s, 3.5 MB/s 2026-02-13T02:20:28.335 DEBUG:teuthology.orchestra.run.trial028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-13T02:20:28.393 DEBUG:teuthology.orchestra.run.trial028:> stat /dev/vg_nvme/lv_3 2026-02-13T02:20:28.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:28 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/978564177' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2026-02-13T02:20:28.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:28 trial027 ceph-mon[24567]: Standby manager daemon z started 2026-02-13T02:20:28.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:28 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:28.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:28 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:28.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:28 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:28.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:28 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:28.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:28 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:28.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:28 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:28.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:28 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:28.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:28 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:28.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:28 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:28.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:28 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:28.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:28 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:28.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:28 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:28.450 INFO:teuthology.orchestra.run.trial028.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-13T02:20:28.450 INFO:teuthology.orchestra.run.trial028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-13T02:20:28.450 INFO:teuthology.orchestra.run.trial028.stdout:Device: 6h/6d Inode: 1314 Links: 1 2026-02-13T02:20:28.451 INFO:teuthology.orchestra.run.trial028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-13T02:20:28.451 INFO:teuthology.orchestra.run.trial028.stdout:Context: system_u:object_r:device_t:s0 2026-02-13T02:20:28.451 INFO:teuthology.orchestra.run.trial028.stdout:Access: 2026-02-13 02:20:27.341969616 +0000 2026-02-13T02:20:28.451 INFO:teuthology.orchestra.run.trial028.stdout:Modify: 2026-02-13 02:18:20.123110135 +0000 2026-02-13T02:20:28.451 INFO:teuthology.orchestra.run.trial028.stdout:Change: 2026-02-13 02:18:20.123110135 +0000 2026-02-13T02:20:28.451 INFO:teuthology.orchestra.run.trial028.stdout: Birth: 2026-02-13 02:18:20.123110135 +0000 2026-02-13T02:20:28.451 DEBUG:teuthology.orchestra.run.trial028:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-13T02:20:28.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:28 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/978564177' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2026-02-13T02:20:28.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:28 trial028 ceph-mon[26701]: Standby manager daemon z started 2026-02-13T02:20:28.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:28 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:28.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:28 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:28.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:28 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:28.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:28 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:28.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:28 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:28.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:28 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:28.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:28 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:28.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:28 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:28.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:28 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:28.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:28 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:28.495 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:28 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:28.495 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:28 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:28.495 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:28 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:28.133+0000 7f6f54b64fc0 -1 mgr[py] Loading module selftest in main interpreter 2026-02-13T02:20:28.495 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:28 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:28.136+0000 7f6f54b64fc0 -1 mgr[py] Loading module test_orchestrator in main interpreter 2026-02-13T02:20:28.495 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:28 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:28.137+0000 7f6f54b64fc0 -1 mgr[py] Loading module osd_support in main interpreter 2026-02-13T02:20:28.495 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:28 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:28.138+0000 7f6f54b64fc0 -1 mgr[py] Loading module orchestrator in main interpreter 2026-02-13T02:20:28.495 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:28 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:28.138+0000 7f6f54b64fc0 -1 mgr[py] Loading module cephadm in main interpreter 2026-02-13T02:20:28.495 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:28 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:28.283+0000 7f6f54b64fc0 -1 mgr[py] Loading module volumes in main interpreter 2026-02-13T02:20:28.495 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:28 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:28.312+0000 7f6f54b64fc0 -1 mgr[py] Loading module k8sevents in main interpreter 2026-02-13T02:20:28.495 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:28 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:28.317+0000 7f6f54b64fc0 -1 mgr[py] Loading module mds_autoscaler in main interpreter 2026-02-13T02:20:28.496 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:28 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:28.317+0000 7f6f54b64fc0 -1 mgr[py] Loading module localpool in main interpreter 2026-02-13T02:20:28.496 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:20:28 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-y[27782]: 2026-02-13T02:20:28.318+0000 7f6f54b64fc0 -1 mgr[py] Loading module influx in main interpreter 2026-02-13T02:20:28.514 INFO:teuthology.orchestra.run.trial028.stderr:1+0 records in 2026-02-13T02:20:28.514 INFO:teuthology.orchestra.run.trial028.stderr:1+0 records out 2026-02-13T02:20:28.514 INFO:teuthology.orchestra.run.trial028.stderr:512 bytes copied, 0.000165758 s, 3.1 MB/s 2026-02-13T02:20:28.515 DEBUG:teuthology.orchestra.run.trial028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-13T02:20:28.573 DEBUG:teuthology.orchestra.run.trial028:> stat /dev/vg_nvme/lv_4 2026-02-13T02:20:28.628 INFO:teuthology.orchestra.run.trial028.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-13T02:20:28.629 INFO:teuthology.orchestra.run.trial028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-13T02:20:28.629 INFO:teuthology.orchestra.run.trial028.stdout:Device: 6h/6d Inode: 1312 Links: 1 2026-02-13T02:20:28.629 INFO:teuthology.orchestra.run.trial028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-13T02:20:28.629 INFO:teuthology.orchestra.run.trial028.stdout:Context: system_u:object_r:device_t:s0 2026-02-13T02:20:28.629 INFO:teuthology.orchestra.run.trial028.stdout:Access: 2026-02-13 02:20:27.341969616 +0000 2026-02-13T02:20:28.629 INFO:teuthology.orchestra.run.trial028.stdout:Modify: 2026-02-13 02:18:20.123110135 +0000 2026-02-13T02:20:28.629 INFO:teuthology.orchestra.run.trial028.stdout:Change: 2026-02-13 02:18:20.123110135 +0000 2026-02-13T02:20:28.629 INFO:teuthology.orchestra.run.trial028.stdout: Birth: 2026-02-13 02:18:20.123110135 +0000 2026-02-13T02:20:28.630 DEBUG:teuthology.orchestra.run.trial028:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-13T02:20:28.691 INFO:teuthology.orchestra.run.trial028.stderr:1+0 records in 2026-02-13T02:20:28.691 INFO:teuthology.orchestra.run.trial028.stderr:1+0 records out 2026-02-13T02:20:28.691 INFO:teuthology.orchestra.run.trial028.stderr:512 bytes copied, 0.000152363 s, 3.4 MB/s 2026-02-13T02:20:28.692 DEBUG:teuthology.orchestra.run.trial028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-13T02:20:28.749 DEBUG:teuthology.orchestra.run.trial107:> set -ex 2026-02-13T02:20:28.749 DEBUG:teuthology.orchestra.run.trial107:> dd if=/scratch_devs of=/dev/stdout 2026-02-13T02:20:28.767 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-13T02:20:28.767 DEBUG:teuthology.orchestra.run.trial107:> stat /dev/vg_nvme/lv_1 2026-02-13T02:20:28.825 INFO:teuthology.orchestra.run.trial107.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-13T02:20:28.825 INFO:teuthology.orchestra.run.trial107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-13T02:20:28.825 INFO:teuthology.orchestra.run.trial107.stdout:Device: 6h/6d Inode: 1315 Links: 1 2026-02-13T02:20:28.825 INFO:teuthology.orchestra.run.trial107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-13T02:20:28.825 INFO:teuthology.orchestra.run.trial107.stdout:Context: system_u:object_r:device_t:s0 2026-02-13T02:20:28.826 INFO:teuthology.orchestra.run.trial107.stdout:Access: 2026-02-13 02:20:26.516157272 +0000 2026-02-13T02:20:28.826 INFO:teuthology.orchestra.run.trial107.stdout:Modify: 2026-02-13 02:18:20.462453501 +0000 2026-02-13T02:20:28.826 INFO:teuthology.orchestra.run.trial107.stdout:Change: 2026-02-13 02:18:20.462453501 +0000 2026-02-13T02:20:28.826 INFO:teuthology.orchestra.run.trial107.stdout: Birth: 2026-02-13 02:18:20.462453501 +0000 2026-02-13T02:20:28.826 DEBUG:teuthology.orchestra.run.trial107:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-13T02:20:28.890 INFO:teuthology.orchestra.run.trial107.stderr:1+0 records in 2026-02-13T02:20:28.890 INFO:teuthology.orchestra.run.trial107.stderr:1+0 records out 2026-02-13T02:20:28.890 INFO:teuthology.orchestra.run.trial107.stderr:512 bytes copied, 0.000159625 s, 3.2 MB/s 2026-02-13T02:20:28.892 DEBUG:teuthology.orchestra.run.trial107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-13T02:20:28.951 DEBUG:teuthology.orchestra.run.trial107:> stat /dev/vg_nvme/lv_2 2026-02-13T02:20:29.008 INFO:teuthology.orchestra.run.trial107.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-13T02:20:29.008 INFO:teuthology.orchestra.run.trial107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-13T02:20:29.008 INFO:teuthology.orchestra.run.trial107.stdout:Device: 6h/6d Inode: 1308 Links: 1 2026-02-13T02:20:29.008 INFO:teuthology.orchestra.run.trial107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-13T02:20:29.008 INFO:teuthology.orchestra.run.trial107.stdout:Context: system_u:object_r:device_t:s0 2026-02-13T02:20:29.008 INFO:teuthology.orchestra.run.trial107.stdout:Access: 2026-02-13 02:20:26.517157246 +0000 2026-02-13T02:20:29.008 INFO:teuthology.orchestra.run.trial107.stdout:Modify: 2026-02-13 02:18:20.462453501 +0000 2026-02-13T02:20:29.008 INFO:teuthology.orchestra.run.trial107.stdout:Change: 2026-02-13 02:18:20.462453501 +0000 2026-02-13T02:20:29.009 INFO:teuthology.orchestra.run.trial107.stdout: Birth: 2026-02-13 02:18:20.462453501 +0000 2026-02-13T02:20:29.009 DEBUG:teuthology.orchestra.run.trial107:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-13T02:20:29.071 INFO:teuthology.orchestra.run.trial107.stderr:1+0 records in 2026-02-13T02:20:29.071 INFO:teuthology.orchestra.run.trial107.stderr:1+0 records out 2026-02-13T02:20:29.071 INFO:teuthology.orchestra.run.trial107.stderr:512 bytes copied, 0.000150068 s, 3.4 MB/s 2026-02-13T02:20:29.072 DEBUG:teuthology.orchestra.run.trial107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-13T02:20:29.129 DEBUG:teuthology.orchestra.run.trial107:> stat /dev/vg_nvme/lv_3 2026-02-13T02:20:29.186 INFO:teuthology.orchestra.run.trial107.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-13T02:20:29.186 INFO:teuthology.orchestra.run.trial107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-13T02:20:29.186 INFO:teuthology.orchestra.run.trial107.stdout:Device: 6h/6d Inode: 1320 Links: 1 2026-02-13T02:20:29.186 INFO:teuthology.orchestra.run.trial107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-13T02:20:29.186 INFO:teuthology.orchestra.run.trial107.stdout:Context: system_u:object_r:device_t:s0 2026-02-13T02:20:29.186 INFO:teuthology.orchestra.run.trial107.stdout:Access: 2026-02-13 02:20:26.517157246 +0000 2026-02-13T02:20:29.186 INFO:teuthology.orchestra.run.trial107.stdout:Modify: 2026-02-13 02:18:20.462453501 +0000 2026-02-13T02:20:29.186 INFO:teuthology.orchestra.run.trial107.stdout:Change: 2026-02-13 02:18:20.462453501 +0000 2026-02-13T02:20:29.186 INFO:teuthology.orchestra.run.trial107.stdout: Birth: 2026-02-13 02:18:20.462453501 +0000 2026-02-13T02:20:29.187 DEBUG:teuthology.orchestra.run.trial107:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-13T02:20:29.249 INFO:teuthology.orchestra.run.trial107.stderr:1+0 records in 2026-02-13T02:20:29.249 INFO:teuthology.orchestra.run.trial107.stderr:1+0 records out 2026-02-13T02:20:29.249 INFO:teuthology.orchestra.run.trial107.stderr:512 bytes copied, 0.000166659 s, 3.1 MB/s 2026-02-13T02:20:29.250 DEBUG:teuthology.orchestra.run.trial107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-13T02:20:29.307 DEBUG:teuthology.orchestra.run.trial107:> stat /dev/vg_nvme/lv_4 2026-02-13T02:20:29.363 INFO:teuthology.orchestra.run.trial107.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-13T02:20:29.364 INFO:teuthology.orchestra.run.trial107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-13T02:20:29.364 INFO:teuthology.orchestra.run.trial107.stdout:Device: 6h/6d Inode: 1300 Links: 1 2026-02-13T02:20:29.364 INFO:teuthology.orchestra.run.trial107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-13T02:20:29.364 INFO:teuthology.orchestra.run.trial107.stdout:Context: system_u:object_r:device_t:s0 2026-02-13T02:20:29.364 INFO:teuthology.orchestra.run.trial107.stdout:Access: 2026-02-13 02:20:26.517157246 +0000 2026-02-13T02:20:29.364 INFO:teuthology.orchestra.run.trial107.stdout:Modify: 2026-02-13 02:18:20.462453501 +0000 2026-02-13T02:20:29.364 INFO:teuthology.orchestra.run.trial107.stdout:Change: 2026-02-13 02:18:20.462453501 +0000 2026-02-13T02:20:29.364 INFO:teuthology.orchestra.run.trial107.stdout: Birth: 2026-02-13 02:18:20.462453501 +0000 2026-02-13T02:20:29.365 DEBUG:teuthology.orchestra.run.trial107:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-13T02:20:29.407 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:29 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2026-02-13T02:20:29.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:29 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2026-02-13T02:20:29.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:29 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:29.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:29 trial027 ceph-mon[24567]: mgrmap e9: x(active, since 31s), standbys: z 2026-02-13T02:20:29.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:29 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2026-02-13T02:20:29.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:29 trial027 ceph-mon[24567]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:29.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:29 trial027 ceph-mon[24567]: Standby manager daemon y started 2026-02-13T02:20:29.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:29 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:29.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:29 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:29.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:29 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:29.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:29 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:29.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:29 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:29.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:29 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:29.424 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:29 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2026-02-13T02:20:29.425 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:29 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2026-02-13T02:20:29.425 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:29 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:29.425 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:29 trial107 ceph-mon[27422]: mgrmap e9: x(active, since 31s), standbys: z 2026-02-13T02:20:29.425 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:29 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2026-02-13T02:20:29.425 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:29 trial107 ceph-mon[27422]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:29.425 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:29 trial107 ceph-mon[27422]: Standby manager daemon y started 2026-02-13T02:20:29.425 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:29 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:29.425 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:29 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:29.425 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:29 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:29.425 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:29 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:29.425 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:29 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:29.426 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:29 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:29.426 INFO:teuthology.orchestra.run.trial107.stderr:1+0 records in 2026-02-13T02:20:29.426 INFO:teuthology.orchestra.run.trial107.stderr:1+0 records out 2026-02-13T02:20:29.426 INFO:teuthology.orchestra.run.trial107.stderr:512 bytes copied, 0.000153113 s, 3.3 MB/s 2026-02-13T02:20:29.428 DEBUG:teuthology.orchestra.run.trial107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-13T02:20:29.486 INFO:tasks.cephadm:Deploying osd.0 on trial027 with /dev/vg_nvme/lv_4... 2026-02-13T02:20:29.486 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_4 2026-02-13T02:20:29.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:29 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2026-02-13T02:20:29.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:29 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2026-02-13T02:20:29.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:29 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:29.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:29 trial028 ceph-mon[26701]: mgrmap e9: x(active, since 31s), standbys: z 2026-02-13T02:20:29.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:29 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2026-02-13T02:20:29.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:29 trial028 ceph-mon[26701]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:29.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:29 trial028 ceph-mon[26701]: Standby manager daemon y started 2026-02-13T02:20:29.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:29 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:29.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:29 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:29.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:29 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:29.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:29 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:29.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:29 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:29.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:29 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:29.603 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:30.026 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:20:30.042 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph orch daemon add osd trial027:vg_nvme/lv_4 --skip-validation 2026-02-13T02:20:30.160 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:30.277 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.273+0000 7f7bd00186c0 1 -- 10.20.193.27:0/4152155412 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7bc810ddc0 msgr2=0x7f7bc810e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:30.277 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.273+0000 7f7bd00186c0 1 --2- 10.20.193.27:0/4152155412 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7bc810ddc0 0x7f7bc810e230 secure :-1 s=READY pgs=141 gs=1 cs=0 l=1 c_cookie=13216059971293692431 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7bbc009970 tx=0x7f7bbc02ed20 comp rx=0 tx=0).stop 2026-02-13T02:20:30.277 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.273+0000 7f7bd00186c0 1 -- 10.20.193.27:0/4152155412 shutdown_connections 2026-02-13T02:20:30.277 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.273+0000 7f7bd00186c0 1 --2- 10.20.193.27:0/4152155412 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f7bc810e770 0x7f7bc816f100 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:30.277 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.273+0000 7f7bd00186c0 1 --2- 10.20.193.27:0/4152155412 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7bc810ddc0 0x7f7bc810e230 unknown :-1 s=CLOSED pgs=141 gs=1 cs=0 l=1 c_cookie=13216059971293692431 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:30.278 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.273+0000 7f7bd00186c0 1 --2- 10.20.193.27:0/4152155412 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7bc810cc40 0x7f7bc810d030 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:30.278 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.273+0000 7f7bd00186c0 1 -- 10.20.193.27:0/4152155412 >> 10.20.193.27:0/4152155412 conn(0x7f7bc80ea360 msgr2=0x7f7bc810a770 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:30.278 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.273+0000 7f7bd00186c0 1 -- 10.20.193.27:0/4152155412 shutdown_connections 2026-02-13T02:20:30.278 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.273+0000 7f7bd00186c0 1 -- 10.20.193.27:0/4152155412 wait complete. 2026-02-13T02:20:30.278 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.273+0000 7f7bd00186c0 1 Processor -- start 2026-02-13T02:20:30.278 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.273+0000 7f7bd00186c0 1 -- start start 2026-02-13T02:20:30.278 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.273+0000 7f7bd00186c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7bc810cc40 0x7f7bc80f6480 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:30.278 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.273+0000 7f7bd00186c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7bc810ddc0 0x7f7bc816f100 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:30.278 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.273+0000 7f7bd00186c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f7bc810e770 0x7f7bc80f6ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:30.278 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.273+0000 7f7bd00186c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc80f7910 con 0x7f7bc810cc40 2026-02-13T02:20:30.278 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.273+0000 7f7bd00186c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc80f7a80 con 0x7f7bc810ddc0 2026-02-13T02:20:30.279 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.273+0000 7f7bd00186c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc80f7c40 con 0x7f7bc810e770 2026-02-13T02:20:30.279 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.273+0000 7f7bcdd896c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7bc810cc40 0x7f7bc80f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:30.279 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.273+0000 7f7bcdd896c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7bc810cc40 0x7f7bc80f6480 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:59968/0 (socket says 10.20.193.27:59968) 2026-02-13T02:20:30.279 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.273+0000 7f7bcdd896c0 1 -- 10.20.193.27:0/355878292 learned_addr learned my addr 10.20.193.27:0/355878292 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:20:30.279 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.274+0000 7f7bcd5886c0 1 --2- 10.20.193.27:0/355878292 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7bc810ddc0 0x7f7bc816f100 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:30.279 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.274+0000 7f7bce58a6c0 1 --2- 10.20.193.27:0/355878292 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f7bc810e770 0x7f7bc80f6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:30.279 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.274+0000 7f7bcdd896c0 1 -- 10.20.193.27:0/355878292 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f7bc810e770 msgr2=0x7f7bc80f6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:30.279 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.274+0000 7f7bcdd896c0 1 --2- 10.20.193.27:0/355878292 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f7bc810e770 0x7f7bc80f6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:30.279 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.274+0000 7f7bcdd896c0 1 -- 10.20.193.27:0/355878292 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7bc810ddc0 msgr2=0x7f7bc816f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:30.279 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.274+0000 7f7bcdd896c0 1 --2- 10.20.193.27:0/355878292 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7bc810ddc0 0x7f7bc816f100 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:30.279 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.274+0000 7f7bcdd896c0 1 -- 10.20.193.27:0/355878292 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bbc009590 con 0x7f7bc810cc40 2026-02-13T02:20:30.280 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.274+0000 7f7bcdd896c0 1 --2- 10.20.193.27:0/355878292 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7bc810cc40 0x7f7bc80f6480 secure :-1 s=READY pgs=142 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7bb800b710 tx=0x7f7bb800bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:30.280 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.274+0000 7f7bb6ffd6c0 1 -- 10.20.193.27:0/355878292 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f7bb8004250 con 0x7f7bc810cc40 2026-02-13T02:20:30.280 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.274+0000 7f7bb6ffd6c0 1 -- 10.20.193.27:0/355878292 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f7bb8002d80 con 0x7f7bc810cc40 2026-02-13T02:20:30.280 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.274+0000 7f7bd00186c0 1 -- 10.20.193.27:0/355878292 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bc80f7f20 con 0x7f7bc810cc40 2026-02-13T02:20:30.280 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.274+0000 7f7bb6ffd6c0 1 -- 10.20.193.27:0/355878292 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f7bb800cdc0 con 0x7f7bc810cc40 2026-02-13T02:20:30.280 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.274+0000 7f7bd00186c0 1 -- 10.20.193.27:0/355878292 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bc80f8450 con 0x7f7bc810cc40 2026-02-13T02:20:30.280 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.274+0000 7f7bd00186c0 1 -- 10.20.193.27:0/355878292 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bc8172fe0 con 0x7f7bc810cc40 2026-02-13T02:20:30.280 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.275+0000 7f7bb6ffd6c0 1 -- 10.20.193.27:0/355878292 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 10) ==== 157010+0+0 (secure 0 0 0) 0x7f7bb80028c0 con 0x7f7bc810cc40 2026-02-13T02:20:30.281 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.276+0000 7f7bb6ffd6c0 1 --2- 10.20.193.27:0/355878292 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f7ba40b55f0 0x7f7ba40d5aa0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:30.282 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.276+0000 7f7bb6ffd6c0 1 -- 10.20.193.27:0/355878292 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 935+0+0 (secure 0 0 0) 0x7f7bb80e1e40 con 0x7f7bc810cc40 2026-02-13T02:20:30.282 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.276+0000 7f7bcd5886c0 1 --2- 10.20.193.27:0/355878292 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f7ba40b55f0 0x7f7ba40d5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:30.282 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.276+0000 7f7bcd5886c0 1 --2- 10.20.193.27:0/355878292 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f7ba40b55f0 0x7f7ba40d5aa0 secure :-1 s=READY pgs=77 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7bbc02fa20 tx=0x7f7bbc009240 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:30.282 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.276+0000 7f7bb6ffd6c0 1 -- 10.20.193.27:0/355878292 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f7bb80a6550 con 0x7f7bc810cc40 2026-02-13T02:20:30.375 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:30.374+0000 7f7bd00186c0 1 -- 10.20.193.27:0/355878292 --> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "trial027:vg_nvme/lv_4", "skip_validation": true, "target": ["mon-mgr", ""]}) -- 0x7f7bc80ed060 con 0x7f7ba40b55f0 2026-02-13T02:20:30.657 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:30 trial027 ceph-mon[24567]: mgrmap e10: x(active, since 32s), standbys: z, y 2026-02-13T02:20:30.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:30 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2026-02-13T02:20:30.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:30 trial028 ceph-mon[26701]: mgrmap e10: x(active, since 32s), standbys: z, y 2026-02-13T02:20:30.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:30 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2026-02-13T02:20:30.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:30 trial107 ceph-mon[27422]: mgrmap e10: x(active, since 32s), standbys: z, y 2026-02-13T02:20:30.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:30 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2026-02-13T02:20:31.624 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:31 trial027 ceph-mon[24567]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:31.624 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:31 trial027 ceph-mon[24567]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial027:vg_nvme/lv_4", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:20:31.624 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:31 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:31.624 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:31 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:31.624 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:31 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:31.624 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:31 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:31.624 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:31 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:31.624 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:31 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:31.624 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:31 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:31.624 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:31 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:31.624 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:31 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:31.624 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:31 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:31.624 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:31 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:31.624 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:31 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:31.624 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:31 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:31.624 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:31 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:31.624 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:31 trial027 ceph-mon[24567]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:20:31.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:31 trial028 ceph-mon[26701]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:31.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:31 trial028 ceph-mon[26701]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial027:vg_nvme/lv_4", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:20:31.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:31 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:31.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:31 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:31.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:31 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:31.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:31 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:31.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:31 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:31.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:31 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:31.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:31 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:31.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:31 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:31.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:31 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:31.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:31 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:31.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:31 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:31.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:31 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:31.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:31 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:31.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:31 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:31.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:31 trial028 ceph-mon[26701]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:20:31.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:31 trial107 ceph-mon[27422]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:31.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:31 trial107 ceph-mon[27422]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial027:vg_nvme/lv_4", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:20:31.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:31 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:31.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:31 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:31.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:31 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:31.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:31 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:31.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:31 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:31.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:31 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:31.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:31 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:31.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:31 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:31.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:31 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:31.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:31 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:31.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:31 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:31.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:31 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:31.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:31 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:31.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:31 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:31.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:31 trial107 ceph-mon[27422]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:20:32.644 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:32 trial028 ceph-mon[26701]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:32.645 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:32 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/372133208' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1a7af29f-2999-4be3-a38e-3fc7feb9f2f5"} : dispatch 2026-02-13T02:20:32.645 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:32 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/372133208' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a7af29f-2999-4be3-a38e-3fc7feb9f2f5"}]': finished 2026-02-13T02:20:32.645 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:32 trial028 ceph-mon[26701]: osdmap e4: 1 total, 0 up, 1 in 2026-02-13T02:20:32.645 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:32 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-13T02:20:32.645 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:32 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:32.645 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:32 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/591900886' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:20:32.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:32 trial027 ceph-mon[24567]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:32.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:32 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/372133208' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1a7af29f-2999-4be3-a38e-3fc7feb9f2f5"} : dispatch 2026-02-13T02:20:32.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:32 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/372133208' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a7af29f-2999-4be3-a38e-3fc7feb9f2f5"}]': finished 2026-02-13T02:20:32.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:32 trial027 ceph-mon[24567]: osdmap e4: 1 total, 0 up, 1 in 2026-02-13T02:20:32.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:32 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-13T02:20:32.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:32 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:32.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:32 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/591900886' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:20:32.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:32 trial107 ceph-mon[27422]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:32.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:32 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/372133208' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1a7af29f-2999-4be3-a38e-3fc7feb9f2f5"} : dispatch 2026-02-13T02:20:32.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:32 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/372133208' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a7af29f-2999-4be3-a38e-3fc7feb9f2f5"}]': finished 2026-02-13T02:20:32.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:32 trial107 ceph-mon[27422]: osdmap e4: 1 total, 0 up, 1 in 2026-02-13T02:20:32.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:32 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-13T02:20:32.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:32 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:32.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:32 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/591900886' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:20:34.421 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:34 trial027 ceph-mon[24567]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:34.422 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:34 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2026-02-13T02:20:34.422 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:34 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:34.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:34 trial028 ceph-mon[26701]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:34.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:34 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2026-02-13T02:20:34.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:34 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:34.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:34 trial107 ceph-mon[27422]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:34.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:34 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2026-02-13T02:20:34.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:34 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:36.592 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:36 trial027 ceph-mon[24567]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:36.592 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:36 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:36.592 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:36 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:36.592 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:36 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:36.593 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:36 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:36.593 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:36 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:36.593 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:36 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:36.593 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:36 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:36.593 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:36 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:36.593 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:36 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:36.593 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:36 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:36.593 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:36 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:36.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:36 trial028 ceph-mon[26701]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:36.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:36 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:36.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:36 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:36.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:36 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:36.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:36 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:36.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:36 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:36.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:36 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:36.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:36 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:36.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:36 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:36.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:36 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:36.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:36 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:36.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:36 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:36.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:36 trial107 ceph-mon[27422]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:36.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:36 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:36.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:36 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:36.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:36 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:36.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:36 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:36.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:36 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:36.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:36 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:36.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:36 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:36.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:36 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:36.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:36 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:36.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:36 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:36.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:36 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:36.784 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:36.781+0000 7f7bb6ffd6c0 1 -- 10.20.193.27:0/355878292 <== mgr.14116 v2:10.20.193.27:6800/1837452956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7f7bc80ed060 con 0x7f7ba40b55f0 2026-02-13T02:20:36.784 INFO:teuthology.orchestra.run.trial027.stdout:Created osd(s) 0 on host 'trial027' 2026-02-13T02:20:36.787 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:36.784+0000 7f7bd00186c0 1 -- 10.20.193.27:0/355878292 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f7ba40b55f0 msgr2=0x7f7ba40d5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:36.787 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:36.784+0000 7f7bd00186c0 1 --2- 10.20.193.27:0/355878292 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f7ba40b55f0 0x7f7ba40d5aa0 secure :-1 s=READY pgs=77 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7bbc02fa20 tx=0x7f7bbc009240 comp rx=0 tx=0).stop 2026-02-13T02:20:36.787 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:36.784+0000 7f7bd00186c0 1 -- 10.20.193.27:0/355878292 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7bc810cc40 msgr2=0x7f7bc80f6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:36.787 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:36.784+0000 7f7bd00186c0 1 --2- 10.20.193.27:0/355878292 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7bc810cc40 0x7f7bc80f6480 secure :-1 s=READY pgs=142 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7bb800b710 tx=0x7f7bb800bbe0 comp rx=0 tx=0).stop 2026-02-13T02:20:36.787 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:36.784+0000 7f7bd00186c0 1 -- 10.20.193.27:0/355878292 shutdown_connections 2026-02-13T02:20:36.787 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:36.784+0000 7f7bd00186c0 1 --2- 10.20.193.27:0/355878292 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f7ba40b55f0 0x7f7ba40d5aa0 unknown :-1 s=CLOSED pgs=77 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:36.787 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:36.784+0000 7f7bd00186c0 1 --2- 10.20.193.27:0/355878292 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f7bc810e770 0x7f7bc80f6ce0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:36.787 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:36.784+0000 7f7bd00186c0 1 --2- 10.20.193.27:0/355878292 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7bc810ddc0 0x7f7bc816f100 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:36.788 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:36.784+0000 7f7bd00186c0 1 --2- 10.20.193.27:0/355878292 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7bc810cc40 0x7f7bc80f6480 unknown :-1 s=CLOSED pgs=142 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:36.788 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:36.784+0000 7f7bd00186c0 1 -- 10.20.193.27:0/355878292 >> 10.20.193.27:0/355878292 conn(0x7f7bc80ea360 msgr2=0x7f7bc80ec950 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:36.788 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:36.784+0000 7f7bd00186c0 1 -- 10.20.193.27:0/355878292 shutdown_connections 2026-02-13T02:20:36.788 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:36.784+0000 7f7bd00186c0 1 -- 10.20.193.27:0/355878292 wait complete. 2026-02-13T02:20:36.830 DEBUG:teuthology.orchestra.run.trial027:osd.0> sudo journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.0.service 2026-02-13T02:20:36.832 INFO:tasks.cephadm:Deploying osd.1 on trial028 with /dev/vg_nvme/lv_4... 2026-02-13T02:20:36.832 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_4 2026-02-13T02:20:36.951 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.b/config 2026-02-13T02:20:37.386 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:20:37.403 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph orch daemon add osd trial028:vg_nvme/lv_4 --skip-validation 2026-02-13T02:20:37.521 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.b/config 2026-02-13T02:20:37.636 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:37 trial028 ceph-mon[26701]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:37.636 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:37 trial028 ceph-mon[26701]: from='osd.0 [v2:10.20.193.27:6802/832083434,v1:10.20.193.27:6803/832083434]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2026-02-13T02:20:37.636 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:37 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:37.636 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:37 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:37.636 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:37 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:37.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.643+0000 7f0478d976c0 1 -- 10.20.193.28:0/1452265791 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f047410ddd0 msgr2=0x7f047410e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:37.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.643+0000 7f0478d976c0 1 --2- 10.20.193.28:0/1452265791 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f047410ddd0 0x7f047410e240 secure :-1 s=READY pgs=5 gs=1 cs=0 l=1 c_cookie=15862876076907917864 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f045c009970 tx=0x7f045c02ed20 comp rx=0 tx=0).stop 2026-02-13T02:20:37.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.643+0000 7f0478d976c0 1 -- 10.20.193.28:0/1452265791 shutdown_connections 2026-02-13T02:20:37.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.643+0000 7f0478d976c0 1 --2- 10.20.193.28:0/1452265791 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f047410e780 0x7f047416f110 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:37.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.643+0000 7f0478d976c0 1 --2- 10.20.193.28:0/1452265791 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f047410ddd0 0x7f047410e240 unknown :-1 s=CLOSED pgs=5 gs=1 cs=0 l=1 c_cookie=15862876076907917864 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:37.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.643+0000 7f0478d976c0 1 --2- 10.20.193.28:0/1452265791 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f047410cc50 0x7f047410d040 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:37.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.643+0000 7f0478d976c0 1 -- 10.20.193.28:0/1452265791 >> 10.20.193.28:0/1452265791 conn(0x7f04740ea370 msgr2=0x7f047410a780 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:37.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.643+0000 7f0478d976c0 1 -- 10.20.193.28:0/1452265791 shutdown_connections 2026-02-13T02:20:37.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.643+0000 7f0478d976c0 1 -- 10.20.193.28:0/1452265791 wait complete. 2026-02-13T02:20:37.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.643+0000 7f0478d976c0 1 Processor -- start 2026-02-13T02:20:37.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.643+0000 7f0478d976c0 1 -- start start 2026-02-13T02:20:37.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.643+0000 7f0478d976c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f047410cc50 0x7f0474061ea0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:37.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.643+0000 7f0478d976c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f047410ddd0 0x7f047416f110 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:37.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.643+0000 7f0478d976c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f047410e780 0x7f04740607a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:37.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.643+0000 7f0478d976c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0474060dc0 con 0x7f047410cc50 2026-02-13T02:20:37.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.643+0000 7f0478d976c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0474060f30 con 0x7f047410ddd0 2026-02-13T02:20:37.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.643+0000 7f0478d976c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f04740610f0 con 0x7f047410e780 2026-02-13T02:20:37.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.643+0000 7f0471d746c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f047410ddd0 0x7f047416f110 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:37.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.643+0000 7f0471d746c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f047410ddd0 0x7f047416f110 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:47478/0 (socket says 10.20.193.28:47478) 2026-02-13T02:20:37.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.643+0000 7f0471d746c0 1 -- 10.20.193.28:0/1405665108 learned_addr learned my addr 10.20.193.28:0/1405665108 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-13T02:20:37.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.643+0000 7f0472d766c0 1 --2- 10.20.193.28:0/1405665108 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f047410e780 0x7f04740607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:37.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.643+0000 7f04725756c0 1 --2- 10.20.193.28:0/1405665108 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f047410cc50 0x7f0474061ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:37.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.643+0000 7f0471d746c0 1 -- 10.20.193.28:0/1405665108 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f047410e780 msgr2=0x7f04740607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:37.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.643+0000 7f0471d746c0 1 --2- 10.20.193.28:0/1405665108 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f047410e780 0x7f04740607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:37.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.643+0000 7f0471d746c0 1 -- 10.20.193.28:0/1405665108 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f047410cc50 msgr2=0x7f0474061ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:37.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.643+0000 7f0471d746c0 1 --2- 10.20.193.28:0/1405665108 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f047410cc50 0x7f0474061ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:37.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.644+0000 7f0471d746c0 1 -- 10.20.193.28:0/1405665108 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f045c009590 con 0x7f047410ddd0 2026-02-13T02:20:37.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.644+0000 7f0472d766c0 1 --2- 10.20.193.28:0/1405665108 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f047410e780 0x7f04740607a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:20:37.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.644+0000 7f0471d746c0 1 --2- 10.20.193.28:0/1405665108 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f047410ddd0 0x7f047416f110 secure :-1 s=READY pgs=6 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f045c0092e0 tx=0x7f045c004670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:37.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.644+0000 7f045b7fe6c0 1 -- 10.20.193.28:0/1405665108 <== mon.2 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f045c002ae0 con 0x7f047410ddd0 2026-02-13T02:20:37.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.644+0000 7f0478d976c0 1 -- 10.20.193.28:0/1405665108 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0474061370 con 0x7f047410ddd0 2026-02-13T02:20:37.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.644+0000 7f045b7fe6c0 1 -- 10.20.193.28:0/1405665108 <== mon.2 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f045c002c40 con 0x7f047410ddd0 2026-02-13T02:20:37.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.644+0000 7f045b7fe6c0 1 -- 10.20.193.28:0/1405665108 <== mon.2 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f045c0372f0 con 0x7f047410ddd0 2026-02-13T02:20:37.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.644+0000 7f0478d976c0 1 -- 10.20.193.28:0/1405665108 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04740fa690 con 0x7f047410ddd0 2026-02-13T02:20:37.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.644+0000 7f0478d976c0 1 -- 10.20.193.28:0/1405665108 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0474172fd0 con 0x7f047410ddd0 2026-02-13T02:20:37.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.646+0000 7f045b7fe6c0 1 -- 10.20.193.28:0/1405665108 <== mon.2 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 10) ==== 157010+0+0 (secure 0 0 0) 0x7f045c007930 con 0x7f047410ddd0 2026-02-13T02:20:37.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.646+0000 7f045b7fe6c0 1 --2- 10.20.193.28:0/1405665108 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f044c0b55f0 0x7f044c0d5aa0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:37.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.646+0000 7f04725756c0 1 --2- 10.20.193.28:0/1405665108 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f044c0b55f0 0x7f044c0d5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:37.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.646+0000 7f045b7fe6c0 1 -- 10.20.193.28:0/1405665108 <== mon.2 v2:10.20.193.28:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1141+0+0 (secure 0 0 0) 0x7f045c1060e0 con 0x7f047410ddd0 2026-02-13T02:20:37.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.647+0000 7f04725756c0 1 --2- 10.20.193.28:0/1405665108 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f044c0b55f0 0x7f044c0d5aa0 secure :-1 s=READY pgs=80 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0468005980 tx=0x7f04680094b0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:37.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.647+0000 7f045b7fe6c0 1 -- 10.20.193.28:0/1405665108 <== mon.2 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f045c0ca720 con 0x7f047410ddd0 2026-02-13T02:20:37.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:37 trial027 ceph-mon[24567]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:37.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:37 trial027 ceph-mon[24567]: from='osd.0 [v2:10.20.193.27:6802/832083434,v1:10.20.193.27:6803/832083434]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2026-02-13T02:20:37.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:37 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:37.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:37 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:37.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:37 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:37.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:37.746+0000 7f0478d976c0 1 -- 10.20.193.28:0/1405665108 --> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_4", "skip_validation": true, "target": ["mon-mgr", ""]}) -- 0x7f04740ed020 con 0x7f044c0b55f0 2026-02-13T02:20:37.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:37 trial107 ceph-mon[27422]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:37.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:37 trial107 ceph-mon[27422]: from='osd.0 [v2:10.20.193.27:6802/832083434,v1:10.20.193.27:6803/832083434]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2026-02-13T02:20:37.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:37 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:37.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:37 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:37.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:37 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:38.629 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:38 trial028 ceph-mon[26701]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:38.629 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:38 trial028 ceph-mon[26701]: from='osd.0 [v2:10.20.193.27:6802/832083434,v1:10.20.193.27:6803/832083434]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2026-02-13T02:20:38.630 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:38 trial028 ceph-mon[26701]: osdmap e5: 1 total, 0 up, 1 in 2026-02-13T02:20:38.630 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:38 trial028 ceph-mon[26701]: from='osd.0 [v2:10.20.193.27:6802/832083434,v1:10.20.193.27:6803/832083434]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.6836, "args": ["host=trial027", "root=default"]} : dispatch 2026-02-13T02:20:38.630 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:38 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-13T02:20:38.630 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:38 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:38.630 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:38 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:38.630 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:38 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:38.630 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:38 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:38.630 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:38 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:38.630 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:38 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:38.630 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:38 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:38.630 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:38 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2026-02-13T02:20:38.631 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:38 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:38.631 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:38 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:38.631 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:38 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:38.631 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:38 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:38.631 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:38 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:38.631 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:38 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:38.631 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:38 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:38.631 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:38 trial028 ceph-mon[26701]: from='osd.0 [v2:10.20.193.27:6802/832083434,v1:10.20.193.27:6803/832083434]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.6836, "args": ["host=trial027", "root=default"]}]': finished 2026-02-13T02:20:38.631 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:38 trial028 ceph-mon[26701]: osdmap e6: 1 total, 0 up, 1 in 2026-02-13T02:20:38.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:38 trial027 ceph-mon[24567]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:38.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:38 trial027 ceph-mon[24567]: from='osd.0 [v2:10.20.193.27:6802/832083434,v1:10.20.193.27:6803/832083434]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2026-02-13T02:20:38.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:38 trial027 ceph-mon[24567]: osdmap e5: 1 total, 0 up, 1 in 2026-02-13T02:20:38.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:38 trial027 ceph-mon[24567]: from='osd.0 [v2:10.20.193.27:6802/832083434,v1:10.20.193.27:6803/832083434]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.6836, "args": ["host=trial027", "root=default"]} : dispatch 2026-02-13T02:20:38.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:38 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-13T02:20:38.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:38 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:38.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:38 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:38.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:38 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:38.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:38 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:38.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:38 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:38.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:38 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:38.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:38 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:38.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:38 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2026-02-13T02:20:38.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:38 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:38.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:38 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:38.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:38 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:38.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:38 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:38.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:38 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:38.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:38 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:38.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:38 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:38.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:38 trial027 ceph-mon[24567]: from='osd.0 [v2:10.20.193.27:6802/832083434,v1:10.20.193.27:6803/832083434]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.6836, "args": ["host=trial027", "root=default"]}]': finished 2026-02-13T02:20:38.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:38 trial027 ceph-mon[24567]: osdmap e6: 1 total, 0 up, 1 in 2026-02-13T02:20:38.661 INFO:journalctl@ceph.osd.0.trial027.stdout:Feb 13 02:20:38 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-0[36849]: 2026-02-13T02:20:38.388+0000 7f8498b1e6c0 -1 osd.0 0 waiting for initial osdmap 2026-02-13T02:20:38.661 INFO:journalctl@ceph.osd.0.trial027.stdout:Feb 13 02:20:38 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-0[36849]: 2026-02-13T02:20:38.390+0000 7f84949e26c0 -1 osd.0 6 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-13T02:20:38.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:38 trial107 ceph-mon[27422]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:38.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:38 trial107 ceph-mon[27422]: from='osd.0 [v2:10.20.193.27:6802/832083434,v1:10.20.193.27:6803/832083434]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2026-02-13T02:20:38.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:38 trial107 ceph-mon[27422]: osdmap e5: 1 total, 0 up, 1 in 2026-02-13T02:20:38.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:38 trial107 ceph-mon[27422]: from='osd.0 [v2:10.20.193.27:6802/832083434,v1:10.20.193.27:6803/832083434]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.6836, "args": ["host=trial027", "root=default"]} : dispatch 2026-02-13T02:20:38.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:38 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-13T02:20:38.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:38 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:38.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:38 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:38.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:38 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:38.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:38 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:38.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:38 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:38.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:38 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:38.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:38 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:38.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:38 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2026-02-13T02:20:38.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:38 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:38.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:38 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:38.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:38 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:38.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:38 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:38.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:38 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:38.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:38 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:38.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:38 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:38.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:38 trial107 ceph-mon[27422]: from='osd.0 [v2:10.20.193.27:6802/832083434,v1:10.20.193.27:6803/832083434]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.6836, "args": ["host=trial027", "root=default"]}]': finished 2026-02-13T02:20:38.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:38 trial107 ceph-mon[27422]: osdmap e6: 1 total, 0 up, 1 in 2026-02-13T02:20:39.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:39 trial027 ceph-mon[24567]: from='client.24128 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_4", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:20:39.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:39 trial027 ceph-mon[24567]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:39.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:39 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-13T02:20:39.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:39 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-13T02:20:39.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:39 trial027 ceph-mon[24567]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8c0ad8cf-4150-43f6-8a9c-258b0ebf6a87"} : dispatch 2026-02-13T02:20:39.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:39 trial027 ceph-mon[24567]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c0ad8cf-4150-43f6-8a9c-258b0ebf6a87"}]': finished 2026-02-13T02:20:39.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:39 trial027 ceph-mon[24567]: osd.0 [v2:10.20.193.27:6802/832083434,v1:10.20.193.27:6803/832083434] boot 2026-02-13T02:20:39.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:39 trial027 ceph-mon[24567]: osdmap e7: 2 total, 1 up, 2 in 2026-02-13T02:20:39.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:39 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-13T02:20:39.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:39 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-13T02:20:39.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:39 trial027 ceph-mon[24567]: from='client.? 10.20.193.28:0/3938327394' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8c0ad8cf-4150-43f6-8a9c-258b0ebf6a87"} : dispatch 2026-02-13T02:20:39.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:39 trial027 ceph-mon[24567]: from='client.? 10.20.193.28:0/1071566020' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:20:39.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:39 trial027 ceph-mon[24567]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:20:39.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:39 trial027 ceph-mon[24567]: Cluster is now healthy 2026-02-13T02:20:39.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:39 trial028 ceph-mon[26701]: from='client.24128 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_4", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:20:39.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:39 trial028 ceph-mon[26701]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:39.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:39 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-13T02:20:39.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:39 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-13T02:20:39.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:39 trial028 ceph-mon[26701]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8c0ad8cf-4150-43f6-8a9c-258b0ebf6a87"} : dispatch 2026-02-13T02:20:39.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:39 trial028 ceph-mon[26701]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c0ad8cf-4150-43f6-8a9c-258b0ebf6a87"}]': finished 2026-02-13T02:20:39.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:39 trial028 ceph-mon[26701]: osd.0 [v2:10.20.193.27:6802/832083434,v1:10.20.193.27:6803/832083434] boot 2026-02-13T02:20:39.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:39 trial028 ceph-mon[26701]: osdmap e7: 2 total, 1 up, 2 in 2026-02-13T02:20:39.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:39 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-13T02:20:39.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:39 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-13T02:20:39.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:39 trial028 ceph-mon[26701]: from='client.? 10.20.193.28:0/3938327394' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8c0ad8cf-4150-43f6-8a9c-258b0ebf6a87"} : dispatch 2026-02-13T02:20:39.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:39 trial028 ceph-mon[26701]: from='client.? 10.20.193.28:0/1071566020' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:20:39.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:39 trial028 ceph-mon[26701]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:20:39.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:39 trial028 ceph-mon[26701]: Cluster is now healthy 2026-02-13T02:20:39.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:39 trial107 ceph-mon[27422]: from='client.24128 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_4", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:20:39.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:39 trial107 ceph-mon[27422]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-13T02:20:39.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:39 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-13T02:20:39.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:39 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-13T02:20:39.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:39 trial107 ceph-mon[27422]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8c0ad8cf-4150-43f6-8a9c-258b0ebf6a87"} : dispatch 2026-02-13T02:20:39.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:39 trial107 ceph-mon[27422]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c0ad8cf-4150-43f6-8a9c-258b0ebf6a87"}]': finished 2026-02-13T02:20:39.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:39 trial107 ceph-mon[27422]: osd.0 [v2:10.20.193.27:6802/832083434,v1:10.20.193.27:6803/832083434] boot 2026-02-13T02:20:39.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:39 trial107 ceph-mon[27422]: osdmap e7: 2 total, 1 up, 2 in 2026-02-13T02:20:39.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:39 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-13T02:20:39.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:39 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-13T02:20:39.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:39 trial107 ceph-mon[27422]: from='client.? 10.20.193.28:0/3938327394' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8c0ad8cf-4150-43f6-8a9c-258b0ebf6a87"} : dispatch 2026-02-13T02:20:39.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:39 trial107 ceph-mon[27422]: from='client.? 10.20.193.28:0/1071566020' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:20:39.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:39 trial107 ceph-mon[27422]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:20:39.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:39 trial107 ceph-mon[27422]: Cluster is now healthy 2026-02-13T02:20:40.555 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:40 trial028 ceph-mon[26701]: purged_snaps scrub starts 2026-02-13T02:20:40.556 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:40 trial028 ceph-mon[26701]: purged_snaps scrub ok 2026-02-13T02:20:40.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:40 trial027 ceph-mon[24567]: purged_snaps scrub starts 2026-02-13T02:20:40.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:40 trial027 ceph-mon[24567]: purged_snaps scrub ok 2026-02-13T02:20:40.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:40 trial107 ceph-mon[27422]: purged_snaps scrub starts 2026-02-13T02:20:40.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:40 trial107 ceph-mon[27422]: purged_snaps scrub ok 2026-02-13T02:20:41.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:41 trial027 ceph-mon[24567]: pgmap v22: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T02:20:41.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:41 trial027 ceph-mon[24567]: pgmap v23: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T02:20:41.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:41 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:41.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:41 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:41.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:41 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:41.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:41 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:41.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:41 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:41.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:41 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:41.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:41 trial027 ceph-mon[24567]: pgmap v24: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T02:20:41.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:41 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:41.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:41 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:41.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:41 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:41.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:41 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2026-02-13T02:20:41.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:41 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:41.693 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:41 trial028 ceph-mon[26701]: pgmap v22: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T02:20:41.693 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:41 trial028 ceph-mon[26701]: pgmap v23: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T02:20:41.693 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:41 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:41.694 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:41 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:41.694 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:41 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:41.694 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:41 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:41.694 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:41 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:41.694 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:41 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:41.694 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:41 trial028 ceph-mon[26701]: pgmap v24: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T02:20:41.694 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:41 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:41.694 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:41 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:41.694 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:41 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:41.694 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:41 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2026-02-13T02:20:41.694 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:41 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:41.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:41 trial107 ceph-mon[27422]: pgmap v22: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T02:20:41.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:41 trial107 ceph-mon[27422]: pgmap v23: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T02:20:41.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:41 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:41.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:41 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:41.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:41 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:41.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:41 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:41.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:41 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:41.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:41 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:41.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:41 trial107 ceph-mon[27422]: pgmap v24: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T02:20:41.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:41 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:41.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:41 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:41.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:41 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:41.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:41 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2026-02-13T02:20:41.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:41 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:42.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:42 trial027 ceph-mon[24567]: pgmap v25: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T02:20:42.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:42 trial027 ceph-mon[24567]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:20:42.677 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:42 trial028 ceph-mon[26701]: pgmap v25: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T02:20:42.677 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:42 trial028 ceph-mon[26701]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:20:42.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:42 trial107 ceph-mon[27422]: pgmap v25: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T02:20:42.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:42 trial107 ceph-mon[27422]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:20:43.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:43 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:43.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:43 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:43.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:43 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:43.726 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:43 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:43.726 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:43 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:43.726 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:43 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:43.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:43 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:43.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:43 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:43.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:43 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:43.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:43.940+0000 7f045b7fe6c0 1 -- 10.20.193.28:0/1405665108 <== mgr.14116 v2:10.20.193.27:6800/1837452956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7f04740ed020 con 0x7f044c0b55f0 2026-02-13T02:20:43.939 INFO:teuthology.orchestra.run.trial028.stdout:Created osd(s) 1 on host 'trial028' 2026-02-13T02:20:43.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:43.943+0000 7f0478d976c0 1 -- 10.20.193.28:0/1405665108 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f044c0b55f0 msgr2=0x7f044c0d5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:43.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:43.943+0000 7f0478d976c0 1 --2- 10.20.193.28:0/1405665108 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f044c0b55f0 0x7f044c0d5aa0 secure :-1 s=READY pgs=80 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0468005980 tx=0x7f04680094b0 comp rx=0 tx=0).stop 2026-02-13T02:20:43.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:43.943+0000 7f0478d976c0 1 -- 10.20.193.28:0/1405665108 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f047410ddd0 msgr2=0x7f047416f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:43.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:43.943+0000 7f0478d976c0 1 --2- 10.20.193.28:0/1405665108 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f047410ddd0 0x7f047416f110 secure :-1 s=READY pgs=6 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f045c0092e0 tx=0x7f045c004670 comp rx=0 tx=0).stop 2026-02-13T02:20:43.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:43.943+0000 7f0478d976c0 1 -- 10.20.193.28:0/1405665108 shutdown_connections 2026-02-13T02:20:43.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:43.943+0000 7f0478d976c0 1 --2- 10.20.193.28:0/1405665108 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f044c0b55f0 0x7f044c0d5aa0 unknown :-1 s=CLOSED pgs=80 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:43.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:43.943+0000 7f0478d976c0 1 --2- 10.20.193.28:0/1405665108 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f047410e780 0x7f04740607a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:43.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:43.943+0000 7f0478d976c0 1 --2- 10.20.193.28:0/1405665108 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f047410ddd0 0x7f047416f110 unknown :-1 s=CLOSED pgs=6 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:43.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:43.943+0000 7f0478d976c0 1 --2- 10.20.193.28:0/1405665108 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f047410cc50 0x7f0474061ea0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:43.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:43.943+0000 7f0478d976c0 1 -- 10.20.193.28:0/1405665108 >> 10.20.193.28:0/1405665108 conn(0x7f04740ea370 msgr2=0x7f04740ec910 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:43.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:43.943+0000 7f0478d976c0 1 -- 10.20.193.28:0/1405665108 shutdown_connections 2026-02-13T02:20:43.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:43.943+0000 7f0478d976c0 1 -- 10.20.193.28:0/1405665108 wait complete. 2026-02-13T02:20:43.984 DEBUG:teuthology.orchestra.run.trial028:osd.1> sudo journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.1.service 2026-02-13T02:20:43.987 INFO:tasks.cephadm:Deploying osd.2 on trial107 with /dev/vg_nvme/lv_4... 2026-02-13T02:20:43.987 DEBUG:teuthology.orchestra.run.trial107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_4 2026-02-13T02:20:44.105 INFO:teuthology.orchestra.run.trial107.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.c/config 2026-02-13T02:20:44.565 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:44 trial107 ceph-mon[27422]: pgmap v26: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T02:20:44.566 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:44 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:44.566 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:44 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:44.566 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:44 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:44.566 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:44 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:44.566 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:44 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:44.566 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:44 trial107 ceph-mon[27422]: pgmap v27: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T02:20:44.566 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:44 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:44.566 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:44 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:44.566 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:44 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:44.567 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:44 trial107 ceph-mon[27422]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2026-02-13T02:20:44.567 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:44 trial107 ceph-mon[27422]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:20:44.567 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:44 trial107 ceph-mon[27422]: Cluster is now healthy 2026-02-13T02:20:44.567 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:44 trial107 ceph-mon[27422]: from='osd.1 [v2:10.20.193.28:6800/4070516162,v1:10.20.193.28:6801/4070516162]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2026-02-13T02:20:44.567 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:44 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:44.567 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:44 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:44.567 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:44 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:44.625 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:20:44.642 DEBUG:teuthology.orchestra.run.trial107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph orch daemon add osd trial107:vg_nvme/lv_4 --skip-validation 2026-02-13T02:20:44.712 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:44 trial028 ceph-mon[26701]: pgmap v26: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T02:20:44.712 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:44 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:44.712 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:44 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:44.712 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:44 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:44.713 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:44 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:44.713 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:44 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:44.713 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:44 trial028 ceph-mon[26701]: pgmap v27: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T02:20:44.713 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:44 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:44.713 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:44 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:44.713 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:44 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:44.713 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:44 trial028 ceph-mon[26701]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2026-02-13T02:20:44.713 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:44 trial028 ceph-mon[26701]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:20:44.713 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:44 trial028 ceph-mon[26701]: Cluster is now healthy 2026-02-13T02:20:44.713 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:44 trial028 ceph-mon[26701]: from='osd.1 [v2:10.20.193.28:6800/4070516162,v1:10.20.193.28:6801/4070516162]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2026-02-13T02:20:44.713 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:44 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:44.714 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:44 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:44.714 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:44 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:44.761 INFO:teuthology.orchestra.run.trial107.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.c/config 2026-02-13T02:20:44.883 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.884+0000 7f41310736c0 1 -- 10.20.193.107:0/1812273775 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f412c117160 msgr2=0x7f412c177af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:44.884 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.884+0000 7f41310736c0 1 --2- 10.20.193.107:0/1812273775 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f412c117160 0x7f412c177af0 secure :-1 s=READY pgs=7 gs=1 cs=0 l=1 c_cookie=8147081940028212688 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f411c006fd0 tx=0x7f411c02fd20 comp rx=0 tx=0).stop 2026-02-13T02:20:44.884 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.884+0000 7f41310736c0 1 -- 10.20.193.107:0/1812273775 shutdown_connections 2026-02-13T02:20:44.884 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.884+0000 7f41310736c0 1 --2- 10.20.193.107:0/1812273775 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f412c117160 0x7f412c177af0 unknown :-1 s=CLOSED pgs=7 gs=1 cs=0 l=1 c_cookie=8147081940028212688 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:44.884 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.884+0000 7f41310736c0 1 --2- 10.20.193.107:0/1812273775 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f412c1167b0 0x7f412c116c20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:44.884 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.884+0000 7f41310736c0 1 --2- 10.20.193.107:0/1812273775 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f412c115630 0x7f412c115a20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:44.884 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.884+0000 7f41310736c0 1 -- 10.20.193.107:0/1812273775 >> 10.20.193.107:0/1812273775 conn(0x7f412c05db80 msgr2=0x7f412c05df80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:44.884 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.884+0000 7f41310736c0 1 -- 10.20.193.107:0/1812273775 shutdown_connections 2026-02-13T02:20:44.884 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.884+0000 7f41310736c0 1 -- 10.20.193.107:0/1812273775 wait complete. 2026-02-13T02:20:44.884 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.884+0000 7f41310736c0 1 Processor -- start 2026-02-13T02:20:44.884 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.884+0000 7f41310736c0 1 -- start start 2026-02-13T02:20:44.885 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.884+0000 7f41310736c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f412c115630 0x7f412c1002a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:44.885 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.884+0000 7f41310736c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f412c1167b0 0x7f412c177ad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:44.885 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.884+0000 7f41310736c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f412c117160 0x7f412c100ae0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:44.885 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.884+0000 7f41310736c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f412c101730 con 0x7f412c1167b0 2026-02-13T02:20:44.885 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.884+0000 7f41310736c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f412c1018a0 con 0x7f412c117160 2026-02-13T02:20:44.885 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.884+0000 7f41310736c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f412c101a60 con 0x7f412c115630 2026-02-13T02:20:44.885 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.884+0000 7f412ad766c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f412c115630 0x7f412c1002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:44.885 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.884+0000 7f412ad766c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f412c115630 0x7f412c1002a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.107:3300/0 says I am v2:10.20.193.107:47050/0 (socket says 10.20.193.107:47050) 2026-02-13T02:20:44.885 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.884+0000 7f412ad766c0 1 -- 10.20.193.107:0/2428486664 learned_addr learned my addr 10.20.193.107:0/2428486664 (peer_addr_for_me v2:10.20.193.107:0/0) 2026-02-13T02:20:44.885 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.884+0000 7f412b5776c0 1 --2- 10.20.193.107:0/2428486664 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f412c117160 0x7f412c100ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:44.885 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.884+0000 7f412a5756c0 1 --2- 10.20.193.107:0/2428486664 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f412c1167b0 0x7f412c177ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:44.886 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.884+0000 7f412ad766c0 1 -- 10.20.193.107:0/2428486664 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f412c117160 msgr2=0x7f412c100ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:44.886 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.884+0000 7f412ad766c0 1 --2- 10.20.193.107:0/2428486664 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f412c117160 0x7f412c100ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:44.886 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.884+0000 7f412ad766c0 1 -- 10.20.193.107:0/2428486664 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f412c1167b0 msgr2=0x7f412c177ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:44.886 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.884+0000 7f412ad766c0 1 --2- 10.20.193.107:0/2428486664 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f412c1167b0 0x7f412c177ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:44.886 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.884+0000 7f412ad766c0 1 -- 10.20.193.107:0/2428486664 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411c00a5b0 con 0x7f412c115630 2026-02-13T02:20:44.886 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.884+0000 7f412ad766c0 1 --2- 10.20.193.107:0/2428486664 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f412c115630 0x7f412c1002a0 secure :-1 s=READY pgs=8 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4120005dc0 tx=0x7f412000bbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:44.886 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.885+0000 7f410bfff6c0 1 -- 10.20.193.107:0/2428486664 <== mon.1 v2:10.20.193.107:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4120004250 con 0x7f412c115630 2026-02-13T02:20:44.886 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.885+0000 7f410bfff6c0 1 -- 10.20.193.107:0/2428486664 <== mon.1 v2:10.20.193.107:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4120009070 con 0x7f412c115630 2026-02-13T02:20:44.886 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.885+0000 7f41310736c0 1 -- 10.20.193.107:0/2428486664 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f412c101ce0 con 0x7f412c115630 2026-02-13T02:20:44.886 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.885+0000 7f410bfff6c0 1 -- 10.20.193.107:0/2428486664 <== mon.1 v2:10.20.193.107:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f412000cd70 con 0x7f412c115630 2026-02-13T02:20:44.886 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.885+0000 7f41310736c0 1 -- 10.20.193.107:0/2428486664 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f412c1021c0 con 0x7f412c115630 2026-02-13T02:20:44.886 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.885+0000 7f41310736c0 1 -- 10.20.193.107:0/2428486664 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f412c17b9d0 con 0x7f412c115630 2026-02-13T02:20:44.888 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.886+0000 7f410bfff6c0 1 -- 10.20.193.107:0/2428486664 <== mon.1 v2:10.20.193.107:3300/0 4 ==== mgrmap(e 10) ==== 157010+0+0 (secure 0 0 0) 0x7f41200028c0 con 0x7f412c115630 2026-02-13T02:20:44.888 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.887+0000 7f410bfff6c0 1 --2- 10.20.193.107:0/2428486664 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f41000b55f0 0x7f41000d5aa0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:44.888 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.887+0000 7f412a5756c0 1 --2- 10.20.193.107:0/2428486664 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f41000b55f0 0x7f41000d5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:44.888 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.887+0000 7f410bfff6c0 1 -- 10.20.193.107:0/2428486664 <== mon.1 v2:10.20.193.107:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1681+0+0 (secure 0 0 0) 0x7f41200e2030 con 0x7f412c115630 2026-02-13T02:20:44.888 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.888+0000 7f412a5756c0 1 --2- 10.20.193.107:0/2428486664 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f41000b55f0 0x7f41000d5aa0 secure :-1 s=READY pgs=85 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4114005b40 tx=0x7f4114009310 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:44.888 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.888+0000 7f410bfff6c0 1 -- 10.20.193.107:0/2428486664 <== mon.1 v2:10.20.193.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f41200a6450 con 0x7f412c115630 2026-02-13T02:20:44.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:44 trial027 ceph-mon[24567]: pgmap v26: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T02:20:44.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:44 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:44.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:44 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:44.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:44 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:44.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:44 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:44.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:44 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:44.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:44 trial027 ceph-mon[24567]: pgmap v27: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T02:20:44.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:44 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:44.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:44 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:44.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:44 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:44.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:44 trial027 ceph-mon[24567]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2026-02-13T02:20:44.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:44 trial027 ceph-mon[24567]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:20:44.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:44 trial027 ceph-mon[24567]: Cluster is now healthy 2026-02-13T02:20:44.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:44 trial027 ceph-mon[24567]: from='osd.1 [v2:10.20.193.28:6800/4070516162,v1:10.20.193.28:6801/4070516162]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2026-02-13T02:20:44.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:44 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:44.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:44 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:44.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:44 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:44.982 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:44.984+0000 7f41310736c0 1 -- 10.20.193.107:0/2428486664 --> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "trial107:vg_nvme/lv_4", "skip_validation": true, "target": ["mon-mgr", ""]}) -- 0x7f412c0f6f20 con 0x7f41000b55f0 2026-02-13T02:20:45.595 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:45 trial107 ceph-mon[27422]: pgmap v28: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T02:20:45.595 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:45 trial107 ceph-mon[27422]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2026-02-13T02:20:45.595 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:45 trial107 ceph-mon[27422]: osdmap e8: 2 total, 1 up, 2 in 2026-02-13T02:20:45.595 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:45 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-13T02:20:45.595 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:45 trial107 ceph-mon[27422]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-13T02:20:45.595 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:45 trial107 ceph-mon[27422]: from='osd.1 [v2:10.20.193.28:6800/4070516162,v1:10.20.193.28:6801/4070516162]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-13T02:20:45.595 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:45 trial107 ceph-mon[27422]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:20:45.595 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:45 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:45.595 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:45 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:45.595 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:45 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:45.595 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:45 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:45.595 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:45 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:45.595 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:45 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:45.595 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:45 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:45.595 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:45 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2026-02-13T02:20:45.595 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:45 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:45.595 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:45 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:45.595 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:45 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:45.595 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:45 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:45.595 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:45 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:45.595 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:45 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:45.595 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:45 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:45.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:45 trial028 ceph-mon[26701]: pgmap v28: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T02:20:45.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:45 trial028 ceph-mon[26701]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2026-02-13T02:20:45.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:45 trial028 ceph-mon[26701]: osdmap e8: 2 total, 1 up, 2 in 2026-02-13T02:20:45.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:45 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-13T02:20:45.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:45 trial028 ceph-mon[26701]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-13T02:20:45.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:45 trial028 ceph-mon[26701]: from='osd.1 [v2:10.20.193.28:6800/4070516162,v1:10.20.193.28:6801/4070516162]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-13T02:20:45.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:45 trial028 ceph-mon[26701]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:20:45.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:45 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:45.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:45 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:45.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:45 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:45.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:45 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:45.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:45 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:45.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:45 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:45.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:45 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:45.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:45 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2026-02-13T02:20:45.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:45 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:45.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:45 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:45.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:45 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:45.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:45 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:45.746 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:45 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:45.746 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:45 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:45.746 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:45 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:45.746 INFO:journalctl@ceph.osd.1.trial028.stdout:Feb 13 02:20:45 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-1[31494]: 2026-02-13T02:20:45.430+0000 7fe0f31496c0 -1 osd.1 0 waiting for initial osdmap 2026-02-13T02:20:45.746 INFO:journalctl@ceph.osd.1.trial028.stdout:Feb 13 02:20:45 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-1[31494]: 2026-02-13T02:20:45.432+0000 7fe0ef00d6c0 -1 osd.1 9 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-13T02:20:45.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:45 trial027 ceph-mon[24567]: pgmap v28: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T02:20:45.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:45 trial027 ceph-mon[24567]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2026-02-13T02:20:45.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:45 trial027 ceph-mon[24567]: osdmap e8: 2 total, 1 up, 2 in 2026-02-13T02:20:45.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:45 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-13T02:20:45.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:45 trial027 ceph-mon[24567]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-13T02:20:45.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:45 trial027 ceph-mon[24567]: from='osd.1 [v2:10.20.193.28:6800/4070516162,v1:10.20.193.28:6801/4070516162]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-13T02:20:45.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:45 trial027 ceph-mon[24567]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:20:45.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:45 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:45.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:45 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:45.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:45 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:45.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:45 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:45.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:45 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:45.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:45 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:45.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:45 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:45.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:45 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2026-02-13T02:20:45.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:45 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:45.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:45 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:45.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:45 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:45.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:45 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:45.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:45 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:45.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:45 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:45.911 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:45 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:46.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:46 trial028 ceph-mon[26701]: from='client.24145 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial107:vg_nvme/lv_4", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:20:46.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:46 trial028 ceph-mon[26701]: pgmap v30: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T02:20:46.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:46 trial028 ceph-mon[26701]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.6836, "args": ["host=trial028", "root=default"]}]': finished 2026-02-13T02:20:46.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:46 trial028 ceph-mon[26701]: osdmap e9: 2 total, 1 up, 2 in 2026-02-13T02:20:46.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:46 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-13T02:20:46.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:46 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-13T02:20:46.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:46 trial028 ceph-mon[26701]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a7813078-94d6-4559-a89a-5525161ab5fd"} : dispatch 2026-02-13T02:20:46.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:46 trial028 ceph-mon[26701]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7813078-94d6-4559-a89a-5525161ab5fd"}]': finished 2026-02-13T02:20:46.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:46 trial028 ceph-mon[26701]: osd.1 [v2:10.20.193.28:6800/4070516162,v1:10.20.193.28:6801/4070516162] boot 2026-02-13T02:20:46.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:46 trial028 ceph-mon[26701]: osdmap e10: 3 total, 2 up, 3 in 2026-02-13T02:20:46.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:46 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-13T02:20:46.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:46 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-13T02:20:46.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:46 trial028 ceph-mon[26701]: from='client.? 10.20.193.107:0/3452485778' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a7813078-94d6-4559-a89a-5525161ab5fd"} : dispatch 2026-02-13T02:20:46.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:46 trial028 ceph-mon[26701]: from='client.? 10.20.193.107:0/1995515744' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:20:46.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:46 trial028 ceph-mon[26701]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:20:46.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:46 trial028 ceph-mon[26701]: Cluster is now healthy 2026-02-13T02:20:46.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:46 trial107 ceph-mon[27422]: from='client.24145 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial107:vg_nvme/lv_4", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:20:46.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:46 trial107 ceph-mon[27422]: pgmap v30: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T02:20:46.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:46 trial107 ceph-mon[27422]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.6836, "args": ["host=trial028", "root=default"]}]': finished 2026-02-13T02:20:46.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:46 trial107 ceph-mon[27422]: osdmap e9: 2 total, 1 up, 2 in 2026-02-13T02:20:46.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:46 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-13T02:20:46.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:46 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-13T02:20:46.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:46 trial107 ceph-mon[27422]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a7813078-94d6-4559-a89a-5525161ab5fd"} : dispatch 2026-02-13T02:20:46.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:46 trial107 ceph-mon[27422]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7813078-94d6-4559-a89a-5525161ab5fd"}]': finished 2026-02-13T02:20:46.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:46 trial107 ceph-mon[27422]: osd.1 [v2:10.20.193.28:6800/4070516162,v1:10.20.193.28:6801/4070516162] boot 2026-02-13T02:20:46.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:46 trial107 ceph-mon[27422]: osdmap e10: 3 total, 2 up, 3 in 2026-02-13T02:20:46.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:46 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-13T02:20:46.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:46 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-13T02:20:46.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:46 trial107 ceph-mon[27422]: from='client.? 10.20.193.107:0/3452485778' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a7813078-94d6-4559-a89a-5525161ab5fd"} : dispatch 2026-02-13T02:20:46.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:46 trial107 ceph-mon[27422]: from='client.? 10.20.193.107:0/1995515744' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:20:46.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:46 trial107 ceph-mon[27422]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:20:46.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:46 trial107 ceph-mon[27422]: Cluster is now healthy 2026-02-13T02:20:46.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:46 trial027 ceph-mon[24567]: from='client.24145 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial107:vg_nvme/lv_4", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:20:46.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:46 trial027 ceph-mon[24567]: pgmap v30: 0 pgs: ; 0 B data, 26 MiB used, 700 GiB / 700 GiB avail 2026-02-13T02:20:46.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:46 trial027 ceph-mon[24567]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.6836, "args": ["host=trial028", "root=default"]}]': finished 2026-02-13T02:20:46.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:46 trial027 ceph-mon[24567]: osdmap e9: 2 total, 1 up, 2 in 2026-02-13T02:20:46.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:46 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-13T02:20:46.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:46 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-13T02:20:46.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:46 trial027 ceph-mon[24567]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a7813078-94d6-4559-a89a-5525161ab5fd"} : dispatch 2026-02-13T02:20:46.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:46 trial027 ceph-mon[24567]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7813078-94d6-4559-a89a-5525161ab5fd"}]': finished 2026-02-13T02:20:46.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:46 trial027 ceph-mon[24567]: osd.1 [v2:10.20.193.28:6800/4070516162,v1:10.20.193.28:6801/4070516162] boot 2026-02-13T02:20:46.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:46 trial027 ceph-mon[24567]: osdmap e10: 3 total, 2 up, 3 in 2026-02-13T02:20:46.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:46 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-13T02:20:46.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:46 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-13T02:20:46.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:46 trial027 ceph-mon[24567]: from='client.? 10.20.193.107:0/3452485778' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a7813078-94d6-4559-a89a-5525161ab5fd"} : dispatch 2026-02-13T02:20:46.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:46 trial027 ceph-mon[24567]: from='client.? 10.20.193.107:0/1995515744' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:20:46.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:46 trial027 ceph-mon[24567]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:20:46.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:46 trial027 ceph-mon[24567]: Cluster is now healthy 2026-02-13T02:20:47.438 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:47 trial107 ceph-mon[27422]: purged_snaps scrub starts 2026-02-13T02:20:47.438 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:47 trial107 ceph-mon[27422]: purged_snaps scrub ok 2026-02-13T02:20:47.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:47 trial028 ceph-mon[26701]: purged_snaps scrub starts 2026-02-13T02:20:47.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:47 trial028 ceph-mon[26701]: purged_snaps scrub ok 2026-02-13T02:20:47.809 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:47 trial027 ceph-mon[24567]: purged_snaps scrub starts 2026-02-13T02:20:47.810 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:47 trial027 ceph-mon[24567]: purged_snaps scrub ok 2026-02-13T02:20:48.522 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:48 trial107 ceph-mon[27422]: pgmap v33: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T02:20:48.522 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:48 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:48.522 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:48 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:48.522 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:48 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:48.522 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:48 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:48.522 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:48 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:48.522 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:48 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:48.523 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:48 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:48.523 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:48 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:48.523 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:48 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:48.523 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:48 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2026-02-13T02:20:48.523 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:48 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:48.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:48 trial028 ceph-mon[26701]: pgmap v33: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T02:20:48.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:48 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:48.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:48 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:48.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:48 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:48.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:48 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:48.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:48 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:48.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:48 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:48.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:48 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:48.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:48 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:48.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:48 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:48.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:48 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2026-02-13T02:20:48.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:48 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:48.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:48 trial027 ceph-mon[24567]: pgmap v33: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T02:20:48.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:48 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:48.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:48 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:48.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:48 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:48.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:48 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:48.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:48 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:48.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:48 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:48.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:48 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:48.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:48 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:48.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:48 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:48.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:48 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2026-02-13T02:20:48.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:48 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:49.697 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:49 trial107 ceph-mon[27422]: pgmap v34: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T02:20:49.697 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:49 trial107 ceph-mon[27422]: pgmap v35: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T02:20:49.697 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:49 trial107 ceph-mon[27422]: pgmap v36: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T02:20:49.697 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:49 trial107 ceph-mon[27422]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:20:49.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:49 trial028 ceph-mon[26701]: pgmap v34: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T02:20:49.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:49 trial028 ceph-mon[26701]: pgmap v35: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T02:20:49.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:49 trial028 ceph-mon[26701]: pgmap v36: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T02:20:49.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:49 trial028 ceph-mon[26701]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:20:49.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:49 trial027 ceph-mon[24567]: pgmap v34: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T02:20:49.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:49 trial027 ceph-mon[24567]: pgmap v35: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T02:20:49.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:49 trial027 ceph-mon[24567]: pgmap v36: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T02:20:49.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:49 trial027 ceph-mon[24567]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:20:50.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:50 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:50.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:50 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:50.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:50 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:50.748 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:50 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:50.748 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:50 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:50.748 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:50 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:50.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:50 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:50.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:50 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:50.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:50 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:50.917 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:50.920+0000 7f410bfff6c0 1 -- 10.20.193.107:0/2428486664 <== mgr.14116 v2:10.20.193.27:6800/1837452956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7f412c0f6f20 con 0x7f41000b55f0 2026-02-13T02:20:50.918 INFO:teuthology.orchestra.run.trial107.stdout:Created osd(s) 2 on host 'trial107' 2026-02-13T02:20:50.920 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:50.922+0000 7f41310736c0 1 -- 10.20.193.107:0/2428486664 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f41000b55f0 msgr2=0x7f41000d5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:50.920 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:50.922+0000 7f41310736c0 1 --2- 10.20.193.107:0/2428486664 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f41000b55f0 0x7f41000d5aa0 secure :-1 s=READY pgs=85 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4114005b40 tx=0x7f4114009310 comp rx=0 tx=0).stop 2026-02-13T02:20:50.920 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:50.922+0000 7f41310736c0 1 -- 10.20.193.107:0/2428486664 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f412c115630 msgr2=0x7f412c1002a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:50.920 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:50.922+0000 7f41310736c0 1 --2- 10.20.193.107:0/2428486664 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f412c115630 0x7f412c1002a0 secure :-1 s=READY pgs=8 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4120005dc0 tx=0x7f412000bbb0 comp rx=0 tx=0).stop 2026-02-13T02:20:50.920 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:50.922+0000 7f41310736c0 1 -- 10.20.193.107:0/2428486664 shutdown_connections 2026-02-13T02:20:50.920 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:50.922+0000 7f41310736c0 1 --2- 10.20.193.107:0/2428486664 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f41000b55f0 0x7f41000d5aa0 unknown :-1 s=CLOSED pgs=85 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:50.921 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:50.922+0000 7f41310736c0 1 --2- 10.20.193.107:0/2428486664 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f412c117160 0x7f412c100ae0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:50.921 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:50.922+0000 7f41310736c0 1 --2- 10.20.193.107:0/2428486664 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f412c1167b0 0x7f412c177ad0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:50.921 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:50.922+0000 7f41310736c0 1 --2- 10.20.193.107:0/2428486664 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f412c115630 0x7f412c1002a0 unknown :-1 s=CLOSED pgs=8 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:50.921 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:50.922+0000 7f41310736c0 1 -- 10.20.193.107:0/2428486664 >> 10.20.193.107:0/2428486664 conn(0x7f412c05db80 msgr2=0x7f412c0f6810 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:50.921 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:50.922+0000 7f41310736c0 1 -- 10.20.193.107:0/2428486664 shutdown_connections 2026-02-13T02:20:50.921 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:20:50.922+0000 7f41310736c0 1 -- 10.20.193.107:0/2428486664 wait complete. 2026-02-13T02:20:50.960 DEBUG:teuthology.orchestra.run.trial107:osd.2> sudo journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.2.service 2026-02-13T02:20:50.962 INFO:tasks.cephadm:Deploying osd.3 on trial027 with /dev/vg_nvme/lv_3... 2026-02-13T02:20:50.963 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_3 2026-02-13T02:20:51.082 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:51.495 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:51 trial027 ceph-mon[24567]: pgmap v37: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T02:20:51.495 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:51 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:51.496 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:51 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:51.496 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:51 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:51.496 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:51 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:51.496 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:51 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:51.496 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:51 trial027 ceph-mon[24567]: pgmap v38: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T02:20:51.496 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:51 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:51.496 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:51 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:51.496 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:51 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:51.496 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:51 trial027 ceph-mon[24567]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2026-02-13T02:20:51.496 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:51 trial027 ceph-mon[24567]: from='osd.2 [v2:10.20.193.107:6800/4075625103,v1:10.20.193.107:6801/4075625103]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2026-02-13T02:20:51.497 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:51 trial027 ceph-mon[24567]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:20:51.497 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:51 trial027 ceph-mon[24567]: Cluster is now healthy 2026-02-13T02:20:51.497 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:51 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:51.497 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:51 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:51.497 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:51 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:51.524 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:20:51.541 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph orch daemon add osd trial027:vg_nvme/lv_3 --skip-validation 2026-02-13T02:20:51.661 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:20:51.737 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:51 trial107 ceph-mon[27422]: pgmap v37: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T02:20:51.737 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:51 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:51.737 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:51 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:51.737 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:51 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:51.737 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:51 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:51.738 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:51 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:51.738 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:51 trial107 ceph-mon[27422]: pgmap v38: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T02:20:51.738 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:51 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:51.738 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:51 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:51.738 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:51 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:51.738 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:51 trial107 ceph-mon[27422]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2026-02-13T02:20:51.738 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:51 trial107 ceph-mon[27422]: from='osd.2 [v2:10.20.193.107:6800/4075625103,v1:10.20.193.107:6801/4075625103]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2026-02-13T02:20:51.738 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:51 trial107 ceph-mon[27422]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:20:51.738 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:51 trial107 ceph-mon[27422]: Cluster is now healthy 2026-02-13T02:20:51.738 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:51 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:51.738 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:51 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:51.739 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:51 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:51.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:51 trial028 ceph-mon[26701]: pgmap v37: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T02:20:51.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:51 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:51.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:51 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:51.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:51 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:51.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:51 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:51.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:51 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:51.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:51 trial028 ceph-mon[26701]: pgmap v38: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T02:20:51.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:51 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:51.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:51 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:51.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:51 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:51.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:51 trial028 ceph-mon[26701]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2026-02-13T02:20:51.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:51 trial028 ceph-mon[26701]: from='osd.2 [v2:10.20.193.107:6800/4075625103,v1:10.20.193.107:6801/4075625103]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2026-02-13T02:20:51.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:51 trial028 ceph-mon[26701]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:20:51.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:51 trial028 ceph-mon[26701]: Cluster is now healthy 2026-02-13T02:20:51.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:51 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:51.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:51 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:51.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:51 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:51.780 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.775+0000 7fd99b7f76c0 1 -- 10.20.193.27:0/1417377809 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd99410eec0 msgr2=0x7fd99410f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:51.780 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.775+0000 7fd99b7f76c0 1 --2- 10.20.193.27:0/1417377809 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd99410eec0 0x7fd99410f2b0 secure :-1 s=READY pgs=150 gs=1 cs=0 l=1 c_cookie=12194548101275434763 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd984009970 tx=0x7fd98402ed20 comp rx=0 tx=0).stop 2026-02-13T02:20:51.781 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.775+0000 7fd99b7f76c0 1 -- 10.20.193.27:0/1417377809 shutdown_connections 2026-02-13T02:20:51.781 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd99b7f76c0 1 --2- 10.20.193.27:0/1417377809 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd9941109f0 0x7fd994171380 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:51.781 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd99b7f76c0 1 --2- 10.20.193.27:0/1417377809 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd994110040 0x7fd9941104b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:51.781 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd99b7f76c0 1 --2- 10.20.193.27:0/1417377809 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd99410eec0 0x7fd99410f2b0 unknown :-1 s=CLOSED pgs=150 gs=1 cs=0 l=1 c_cookie=12194548101275434763 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:51.781 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd99b7f76c0 1 -- 10.20.193.27:0/1417377809 >> 10.20.193.27:0/1417377809 conn(0x7fd9940ec5c0 msgr2=0x7fd99410c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:51.781 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd99b7f76c0 1 -- 10.20.193.27:0/1417377809 shutdown_connections 2026-02-13T02:20:51.781 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd99b7f76c0 1 -- 10.20.193.27:0/1417377809 wait complete. 2026-02-13T02:20:51.781 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd99b7f76c0 1 Processor -- start 2026-02-13T02:20:51.781 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd99b7f76c0 1 -- start start 2026-02-13T02:20:51.781 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd99b7f76c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd99410eec0 0x7fd9940f8630 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:51.782 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd99b7f76c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd994110040 0x7fd994171380 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:51.782 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd99b7f76c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd9941109f0 0x7fd9940f8e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:51.782 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd99b7f76c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd9940f9ac0 con 0x7fd99410eec0 2026-02-13T02:20:51.782 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd99b7f76c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd9940f9c30 con 0x7fd9941109f0 2026-02-13T02:20:51.782 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd99b7f76c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd9940f9df0 con 0x7fd994110040 2026-02-13T02:20:51.782 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd9995686c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd99410eec0 0x7fd9940f8630 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:51.782 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd9995686c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd99410eec0 0x7fd9940f8630 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:52600/0 (socket says 10.20.193.27:52600) 2026-02-13T02:20:51.782 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd9995686c0 1 -- 10.20.193.27:0/2761513532 learned_addr learned my addr 10.20.193.27:0/2761513532 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:20:51.782 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd998d676c0 1 --2- 10.20.193.27:0/2761513532 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd994110040 0x7fd994171380 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:51.782 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd9995686c0 1 -- 10.20.193.27:0/2761513532 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd994110040 msgr2=0x7fd994171380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:51.782 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd9995686c0 1 --2- 10.20.193.27:0/2761513532 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd994110040 0x7fd994171380 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:51.783 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd9995686c0 1 -- 10.20.193.27:0/2761513532 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd9941109f0 msgr2=0x7fd9940f8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:51.783 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd999d696c0 1 --2- 10.20.193.27:0/2761513532 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd9941109f0 0x7fd9940f8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:51.783 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd9995686c0 1 --2- 10.20.193.27:0/2761513532 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd9941109f0 0x7fd9940f8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:51.783 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd9995686c0 1 -- 10.20.193.27:0/2761513532 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd984009590 con 0x7fd99410eec0 2026-02-13T02:20:51.783 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd999d696c0 1 --2- 10.20.193.27:0/2761513532 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd9941109f0 0x7fd9940f8e90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:20:51.783 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd998d676c0 1 --2- 10.20.193.27:0/2761513532 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd994110040 0x7fd994171380 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:20:51.783 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd9995686c0 1 --2- 10.20.193.27:0/2761513532 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd99410eec0 0x7fd9940f8630 secure :-1 s=READY pgs=151 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd9840092e0 tx=0x7fd984004670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:51.783 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd9827fc6c0 1 -- 10.20.193.27:0/2761513532 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd984002ae0 con 0x7fd99410eec0 2026-02-13T02:20:51.783 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd9827fc6c0 1 -- 10.20.193.27:0/2761513532 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd984002c40 con 0x7fd99410eec0 2026-02-13T02:20:51.783 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd99b7f76c0 1 -- 10.20.193.27:0/2761513532 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9940fa070 con 0x7fd99410eec0 2026-02-13T02:20:51.783 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd9827fc6c0 1 -- 10.20.193.27:0/2761513532 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd9840372f0 con 0x7fd99410eec0 2026-02-13T02:20:51.783 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.776+0000 7fd99b7f76c0 1 -- 10.20.193.27:0/2761513532 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9940fa540 con 0x7fd99410eec0 2026-02-13T02:20:51.784 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.777+0000 7fd99b7f76c0 1 -- 10.20.193.27:0/2761513532 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9941752c0 con 0x7fd99410eec0 2026-02-13T02:20:51.784 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.778+0000 7fd9827fc6c0 1 -- 10.20.193.27:0/2761513532 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 10) ==== 157010+0+0 (secure 0 0 0) 0x7fd984007880 con 0x7fd99410eec0 2026-02-13T02:20:51.784 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.779+0000 7fd9827fc6c0 1 --2- 10.20.193.27:0/2761513532 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd9700b55f0 0x7fd9700d5aa0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:51.784 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.779+0000 7fd9827fc6c0 1 -- 10.20.193.27:0/2761513532 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 2221+0+0 (secure 0 0 0) 0x7fd984106510 con 0x7fd99410eec0 2026-02-13T02:20:51.784 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.779+0000 7fd998d676c0 1 --2- 10.20.193.27:0/2761513532 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd9700b55f0 0x7fd9700d5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:51.784 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.779+0000 7fd998d676c0 1 --2- 10.20.193.27:0/2761513532 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd9700b55f0 0x7fd9700d5aa0 secure :-1 s=READY pgs=90 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd988004750 tx=0x7fd988009260 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:51.784 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.779+0000 7fd9827fc6c0 1 -- 10.20.193.27:0/2761513532 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fd9840ca720 con 0x7fd99410eec0 2026-02-13T02:20:51.875 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:51.874+0000 7fd99b7f76c0 1 -- 10.20.193.27:0/2761513532 --> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "trial027:vg_nvme/lv_3", "skip_validation": true, "target": ["mon-mgr", ""]}) -- 0x7fd99404ca20 con 0x7fd9700b55f0 2026-02-13T02:20:52.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:52 trial107 ceph-mon[27422]: pgmap v39: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T02:20:52.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:52 trial107 ceph-mon[27422]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2026-02-13T02:20:52.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:52 trial107 ceph-mon[27422]: osdmap e11: 3 total, 2 up, 3 in 2026-02-13T02:20:52.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:52 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-13T02:20:52.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:52 trial107 ceph-mon[27422]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.6836, "args": ["host=trial107", "root=default"]} : dispatch 2026-02-13T02:20:52.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:52 trial107 ceph-mon[27422]: from='osd.2 [v2:10.20.193.107:6800/4075625103,v1:10.20.193.107:6801/4075625103]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.6836, "args": ["host=trial107", "root=default"]} : dispatch 2026-02-13T02:20:52.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:52 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:52.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:52 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:52.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:52 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:52.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:52 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:52.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:52 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:52.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:52 trial107 ceph-mon[27422]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:20:52.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:52 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:52.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:52 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:52.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:52 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2026-02-13T02:20:52.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:52 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:52.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:52 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:52.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:52 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:52.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:52 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:52.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:52 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:52.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:52 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:52.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:52 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:52.752 INFO:journalctl@ceph.osd.2.trial107.stdout:Feb 13 02:20:52 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-2[32542]: 2026-02-13T02:20:52.500+0000 7fb4ea37c6c0 -1 osd.2 0 waiting for initial osdmap 2026-02-13T02:20:52.753 INFO:journalctl@ceph.osd.2.trial107.stdout:Feb 13 02:20:52 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-2[32542]: 2026-02-13T02:20:52.503+0000 7fb4e22386c0 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-13T02:20:52.787 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:52 trial027 ceph-mon[24567]: pgmap v39: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T02:20:52.788 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:52 trial027 ceph-mon[24567]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2026-02-13T02:20:52.788 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:52 trial027 ceph-mon[24567]: osdmap e11: 3 total, 2 up, 3 in 2026-02-13T02:20:52.788 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:52 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-13T02:20:52.788 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:52 trial027 ceph-mon[24567]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.6836, "args": ["host=trial107", "root=default"]} : dispatch 2026-02-13T02:20:52.788 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:52 trial027 ceph-mon[24567]: from='osd.2 [v2:10.20.193.107:6800/4075625103,v1:10.20.193.107:6801/4075625103]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.6836, "args": ["host=trial107", "root=default"]} : dispatch 2026-02-13T02:20:52.788 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:52 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:52.788 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:52 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:52.788 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:52 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:52.788 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:52 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:52.788 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:52 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:52.788 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:52 trial027 ceph-mon[24567]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:20:52.788 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:52 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:52.788 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:52 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:52.788 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:52 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2026-02-13T02:20:52.788 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:52 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:52.788 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:52 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:52.788 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:52 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:52.788 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:52 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:52.788 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:52 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:52.788 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:52 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:52.788 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:52 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:52.993 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:52 trial028 ceph-mon[26701]: pgmap v39: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T02:20:52.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:52 trial028 ceph-mon[26701]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2026-02-13T02:20:52.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:52 trial028 ceph-mon[26701]: osdmap e11: 3 total, 2 up, 3 in 2026-02-13T02:20:52.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:52 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-13T02:20:52.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:52 trial028 ceph-mon[26701]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.6836, "args": ["host=trial107", "root=default"]} : dispatch 2026-02-13T02:20:52.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:52 trial028 ceph-mon[26701]: from='osd.2 [v2:10.20.193.107:6800/4075625103,v1:10.20.193.107:6801/4075625103]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.6836, "args": ["host=trial107", "root=default"]} : dispatch 2026-02-13T02:20:52.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:52 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:52.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:52 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:52.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:52 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:52.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:52 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:52.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:52 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:52.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:52 trial028 ceph-mon[26701]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:20:52.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:52 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:52.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:52 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:52.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:52 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2026-02-13T02:20:52.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:52 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:52.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:52 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:52.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:52 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:52.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:52 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:52.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:52 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:52.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:52 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:52.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:52 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:53.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:53 trial107 ceph-mon[27422]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial027:vg_nvme/lv_3", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:20:53.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:53 trial107 ceph-mon[27422]: pgmap v41: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T02:20:53.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:53 trial107 ceph-mon[27422]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.6836, "args": ["host=trial107", "root=default"]}]': finished 2026-02-13T02:20:53.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:53 trial107 ceph-mon[27422]: osdmap e12: 3 total, 2 up, 3 in 2026-02-13T02:20:53.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:53 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-13T02:20:53.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:53 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-13T02:20:53.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:53 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3787897921' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "268f9714-faed-452b-ae34-3240fa23153a"} : dispatch 2026-02-13T02:20:53.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:53 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3787897921' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "268f9714-faed-452b-ae34-3240fa23153a"}]': finished 2026-02-13T02:20:53.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:53 trial107 ceph-mon[27422]: osd.2 [v2:10.20.193.107:6800/4075625103,v1:10.20.193.107:6801/4075625103] boot 2026-02-13T02:20:53.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:53 trial107 ceph-mon[27422]: osdmap e13: 4 total, 3 up, 4 in 2026-02-13T02:20:53.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:53 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-13T02:20:53.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:53 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-13T02:20:53.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:53 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3019806507' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:20:53.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:53 trial107 ceph-mon[27422]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:20:53.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:53 trial107 ceph-mon[27422]: Cluster is now healthy 2026-02-13T02:20:53.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:53 trial027 ceph-mon[24567]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial027:vg_nvme/lv_3", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:20:53.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:53 trial027 ceph-mon[24567]: pgmap v41: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T02:20:53.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:53 trial027 ceph-mon[24567]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.6836, "args": ["host=trial107", "root=default"]}]': finished 2026-02-13T02:20:53.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:53 trial027 ceph-mon[24567]: osdmap e12: 3 total, 2 up, 3 in 2026-02-13T02:20:53.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:53 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-13T02:20:53.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:53 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-13T02:20:53.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:53 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3787897921' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "268f9714-faed-452b-ae34-3240fa23153a"} : dispatch 2026-02-13T02:20:53.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:53 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3787897921' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "268f9714-faed-452b-ae34-3240fa23153a"}]': finished 2026-02-13T02:20:53.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:53 trial027 ceph-mon[24567]: osd.2 [v2:10.20.193.107:6800/4075625103,v1:10.20.193.107:6801/4075625103] boot 2026-02-13T02:20:53.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:53 trial027 ceph-mon[24567]: osdmap e13: 4 total, 3 up, 4 in 2026-02-13T02:20:53.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:53 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-13T02:20:53.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:53 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-13T02:20:53.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:53 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3019806507' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:20:53.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:53 trial027 ceph-mon[24567]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:20:53.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:53 trial027 ceph-mon[24567]: Cluster is now healthy 2026-02-13T02:20:53.993 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:53 trial028 ceph-mon[26701]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial027:vg_nvme/lv_3", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:20:53.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:53 trial028 ceph-mon[26701]: pgmap v41: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-13T02:20:53.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:53 trial028 ceph-mon[26701]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.6836, "args": ["host=trial107", "root=default"]}]': finished 2026-02-13T02:20:53.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:53 trial028 ceph-mon[26701]: osdmap e12: 3 total, 2 up, 3 in 2026-02-13T02:20:53.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:53 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-13T02:20:53.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:53 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-13T02:20:53.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:53 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3787897921' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "268f9714-faed-452b-ae34-3240fa23153a"} : dispatch 2026-02-13T02:20:53.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:53 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3787897921' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "268f9714-faed-452b-ae34-3240fa23153a"}]': finished 2026-02-13T02:20:53.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:53 trial028 ceph-mon[26701]: osd.2 [v2:10.20.193.107:6800/4075625103,v1:10.20.193.107:6801/4075625103] boot 2026-02-13T02:20:53.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:53 trial028 ceph-mon[26701]: osdmap e13: 4 total, 3 up, 4 in 2026-02-13T02:20:53.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:53 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-13T02:20:53.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:53 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-13T02:20:53.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:53 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3019806507' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:20:53.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:53 trial028 ceph-mon[26701]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:20:53.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:53 trial028 ceph-mon[26701]: Cluster is now healthy 2026-02-13T02:20:54.688 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:54 trial027 ceph-mon[24567]: purged_snaps scrub starts 2026-02-13T02:20:54.688 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:54 trial027 ceph-mon[24567]: purged_snaps scrub ok 2026-02-13T02:20:54.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:54 trial107 ceph-mon[27422]: purged_snaps scrub starts 2026-02-13T02:20:54.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:54 trial107 ceph-mon[27422]: purged_snaps scrub ok 2026-02-13T02:20:54.993 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:54 trial028 ceph-mon[26701]: purged_snaps scrub starts 2026-02-13T02:20:54.993 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:54 trial028 ceph-mon[26701]: purged_snaps scrub ok 2026-02-13T02:20:55.815 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:55 trial027 ceph-mon[24567]: pgmap v44: 0 pgs: ; 0 B data, 79 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T02:20:55.815 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:55 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:55.815 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:55 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:55.816 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:55 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:55.816 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:55 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:55.816 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:55 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:55.816 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:55 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:55.816 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:55 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:55.816 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:55 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:55.816 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:55 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:55.816 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:55 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2026-02-13T02:20:55.816 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:55 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2026-02-13T02:20:55.817 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:55 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:55.993 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:55 trial028 ceph-mon[26701]: pgmap v44: 0 pgs: ; 0 B data, 79 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T02:20:55.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:55 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:55.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:55 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:55.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:55 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:55.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:55 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:55.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:55 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:55.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:55 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:55.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:55 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:55.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:55 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:55.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:55 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:55.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:55 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2026-02-13T02:20:55.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:55 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2026-02-13T02:20:55.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:55 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:56.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:55 trial107 ceph-mon[27422]: pgmap v44: 0 pgs: ; 0 B data, 79 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T02:20:56.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:55 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:56.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:55 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:56.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:55 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:56.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:55 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:56.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:55 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:56.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:55 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:56.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:55 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:56.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:55 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:56.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:55 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:56.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:55 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2026-02-13T02:20:56.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:55 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2026-02-13T02:20:56.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:55 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:56.757 INFO:journalctl@ceph.osd.2.trial107.stdout:Feb 13 02:20:56 trial107 sudo[35554]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme2n1 2026-02-13T02:20:56.757 INFO:journalctl@ceph.osd.2.trial107.stdout:Feb 13 02:20:56 trial107 sudo[35554]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-13T02:20:56.757 INFO:journalctl@ceph.osd.2.trial107.stdout:Feb 13 02:20:56 trial107 sudo[35554]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-13T02:20:56.757 INFO:journalctl@ceph.osd.2.trial107.stdout:Feb 13 02:20:56 trial107 sudo[35554]: pam_unix(sudo:session): session closed for user root 2026-02-13T02:20:56.757 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:56 trial107 ceph-mon[27422]: pgmap v45: 0 pgs: ; 0 B data, 79 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T02:20:56.758 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:56 trial107 ceph-mon[27422]: pgmap v46: 0 pgs: ; 0 B data, 79 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T02:20:56.758 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:56 trial107 ceph-mon[27422]: pgmap v47: 0 pgs: ; 0 B data, 79 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T02:20:56.758 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:56 trial107 ceph-mon[27422]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:20:56.758 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:56 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2026-02-13T02:20:56.758 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:56 trial107 ceph-mon[27422]: osdmap e14: 4 total, 3 up, 4 in 2026-02-13T02:20:56.758 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:56 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-13T02:20:56.758 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:56 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2026-02-13T02:20:56.759 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:56 trial027 ceph-mon[24567]: pgmap v45: 0 pgs: ; 0 B data, 79 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T02:20:56.759 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:56 trial027 ceph-mon[24567]: pgmap v46: 0 pgs: ; 0 B data, 79 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T02:20:56.759 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:56 trial027 ceph-mon[24567]: pgmap v47: 0 pgs: ; 0 B data, 79 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T02:20:56.759 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:56 trial027 ceph-mon[24567]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:20:56.759 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:56 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2026-02-13T02:20:56.759 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:56 trial027 ceph-mon[24567]: osdmap e14: 4 total, 3 up, 4 in 2026-02-13T02:20:56.759 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:56 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-13T02:20:56.759 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:56 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2026-02-13T02:20:56.760 INFO:journalctl@ceph.osd.0.trial027.stdout:Feb 13 02:20:56 trial027 sudo[42857]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme2n1 2026-02-13T02:20:56.760 INFO:journalctl@ceph.osd.0.trial027.stdout:Feb 13 02:20:56 trial027 sudo[42857]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-13T02:20:56.760 INFO:journalctl@ceph.osd.0.trial027.stdout:Feb 13 02:20:56 trial027 sudo[42857]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-13T02:20:56.760 INFO:journalctl@ceph.osd.0.trial027.stdout:Feb 13 02:20:56 trial027 sudo[42857]: pam_unix(sudo:session): session closed for user root 2026-02-13T02:20:56.760 INFO:journalctl@ceph.osd.0.trial027.stdout:Feb 13 02:20:56 trial027 sudo[42862]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme mtfdkcc1t9tgp-1bk1jabyy smart-log-add --json /dev/nvme2n1 2026-02-13T02:20:56.760 INFO:journalctl@ceph.osd.0.trial027.stdout:Feb 13 02:20:56 trial027 sudo[42862]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-13T02:20:56.760 INFO:journalctl@ceph.osd.0.trial027.stdout:Feb 13 02:20:56 trial027 sudo[42862]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-13T02:20:56.760 INFO:journalctl@ceph.osd.0.trial027.stdout:Feb 13 02:20:56 trial027 sudo[42862]: pam_unix(sudo:session): session closed for user root 2026-02-13T02:20:56.872 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:56 trial028 ceph-mon[26701]: pgmap v45: 0 pgs: ; 0 B data, 79 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T02:20:56.872 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:56 trial028 ceph-mon[26701]: pgmap v46: 0 pgs: ; 0 B data, 79 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T02:20:56.872 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:56 trial028 ceph-mon[26701]: pgmap v47: 0 pgs: ; 0 B data, 79 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T02:20:56.872 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:56 trial028 ceph-mon[26701]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:20:56.872 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:56 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2026-02-13T02:20:56.872 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:56 trial028 ceph-mon[26701]: osdmap e14: 4 total, 3 up, 4 in 2026-02-13T02:20:56.873 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:56 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-13T02:20:56.873 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:56 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2026-02-13T02:20:56.873 INFO:journalctl@ceph.osd.1.trial028.stdout:Feb 13 02:20:56 trial028 sudo[34631]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme2n1 2026-02-13T02:20:56.873 INFO:journalctl@ceph.osd.1.trial028.stdout:Feb 13 02:20:56 trial028 sudo[34631]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-13T02:20:56.873 INFO:journalctl@ceph.osd.1.trial028.stdout:Feb 13 02:20:56 trial028 sudo[34631]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-13T02:20:56.873 INFO:journalctl@ceph.osd.1.trial028.stdout:Feb 13 02:20:56 trial028 sudo[34631]: pam_unix(sudo:session): session closed for user root 2026-02-13T02:20:56.873 INFO:journalctl@ceph.osd.1.trial028.stdout:Feb 13 02:20:56 trial028 sudo[34635]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme mtfdkcc1t9tgp-1bk1jabyy smart-log-add --json /dev/nvme2n1 2026-02-13T02:20:56.873 INFO:journalctl@ceph.osd.1.trial028.stdout:Feb 13 02:20:56 trial028 sudo[34635]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-13T02:20:56.873 INFO:journalctl@ceph.osd.1.trial028.stdout:Feb 13 02:20:56 trial028 sudo[34635]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-13T02:20:56.874 INFO:journalctl@ceph.osd.1.trial028.stdout:Feb 13 02:20:56 trial028 sudo[34635]: pam_unix(sudo:session): session closed for user root 2026-02-13T02:20:57.008 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:56 trial027 sudo[42880]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme1n1 2026-02-13T02:20:57.008 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:56 trial027 sudo[42880]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-13T02:20:57.008 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:56 trial027 sudo[42880]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-13T02:20:57.008 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:56 trial027 sudo[42880]: pam_unix(sudo:session): session closed for user root 2026-02-13T02:20:57.008 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:56 trial027 sudo[42884]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme mtfdkcc1t9tgp-1bk1jabyy smart-log-add --json /dev/nvme1n1 2026-02-13T02:20:57.009 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:56 trial027 sudo[42884]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-13T02:20:57.009 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:56 trial027 sudo[42884]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-13T02:20:57.009 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:56 trial027 sudo[42884]: pam_unix(sudo:session): session closed for user root 2026-02-13T02:20:57.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:56 trial028 sudo[34639]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme1n1 2026-02-13T02:20:57.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:56 trial028 sudo[34639]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-13T02:20:57.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:56 trial028 sudo[34639]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-13T02:20:57.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:56 trial028 sudo[34639]: pam_unix(sudo:session): session closed for user root 2026-02-13T02:20:57.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:56 trial028 sudo[34643]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme mtfdkcc1t9tgp-1bk1jabyy smart-log-add --json /dev/nvme1n1 2026-02-13T02:20:57.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:56 trial028 sudo[34643]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-13T02:20:57.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:56 trial028 sudo[34643]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-13T02:20:57.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:56 trial028 sudo[34643]: pam_unix(sudo:session): session closed for user root 2026-02-13T02:20:57.250 INFO:journalctl@ceph.osd.2.trial107.stdout:Feb 13 02:20:56 trial107 sudo[35558]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme mtfdkcc1t9tgp-1bk1jabyy smart-log-add --json /dev/nvme2n1 2026-02-13T02:20:57.250 INFO:journalctl@ceph.osd.2.trial107.stdout:Feb 13 02:20:56 trial107 sudo[35558]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-13T02:20:57.250 INFO:journalctl@ceph.osd.2.trial107.stdout:Feb 13 02:20:56 trial107 sudo[35558]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-13T02:20:57.250 INFO:journalctl@ceph.osd.2.trial107.stdout:Feb 13 02:20:56 trial107 sudo[35558]: pam_unix(sudo:session): session closed for user root 2026-02-13T02:20:57.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:56 trial107 sudo[35562]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2026-02-13T02:20:57.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:56 trial107 sudo[35562]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-13T02:20:57.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:56 trial107 sudo[35562]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-13T02:20:57.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:56 trial107 sudo[35562]: pam_unix(sudo:session): session closed for user root 2026-02-13T02:20:57.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:56 trial107 sudo[35566]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme mtfdkcc1t9tgp-1bk1jabyy smart-log-add --json /dev/nvme0n1 2026-02-13T02:20:57.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:56 trial107 sudo[35566]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-13T02:20:57.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:56 trial107 sudo[35566]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-13T02:20:57.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:56 trial107 sudo[35566]: pam_unix(sudo:session): session closed for user root 2026-02-13T02:20:57.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2026-02-13T02:20:57.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:57 trial027 ceph-mon[24567]: osdmap e15: 4 total, 3 up, 4 in 2026-02-13T02:20:57.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-13T02:20:57.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:57.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:57.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:57.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:57 trial027 ceph-mon[24567]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2026-02-13T02:20:57.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:57 trial027 ceph-mon[24567]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2026-02-13T02:20:57.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-13T02:20:57.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-13T02:20:57.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-13T02:20:57.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:57.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-13T02:20:57.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-13T02:20:57.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-13T02:20:57.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:57 trial027 ceph-mon[24567]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2026-02-13T02:20:57.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:57 trial027 ceph-mon[24567]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2026-02-13T02:20:57.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:57.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-13T02:20:57.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-13T02:20:57.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-13T02:20:57.911 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:57 trial027 ceph-mon[24567]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2026-02-13T02:20:57.911 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:57 trial027 ceph-mon[24567]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2026-02-13T02:20:57.911 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:57.911 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:57.911 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:57.911 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:57.911 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:57 trial027 ceph-mon[24567]: pgmap v50: 1 pgs: 1 creating+peering; 0 B data, 79 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T02:20:57.993 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:57 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2026-02-13T02:20:57.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:57 trial028 ceph-mon[26701]: osdmap e15: 4 total, 3 up, 4 in 2026-02-13T02:20:57.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:57 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-13T02:20:57.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:57 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:57.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:57 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:57.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:57 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:57.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:57 trial028 ceph-mon[26701]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2026-02-13T02:20:57.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:57 trial028 ceph-mon[26701]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2026-02-13T02:20:57.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:57 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-13T02:20:57.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:57 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-13T02:20:57.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:57 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-13T02:20:57.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:57 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:57.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:57 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-13T02:20:57.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:57 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-13T02:20:57.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:57 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-13T02:20:57.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:57 trial028 ceph-mon[26701]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2026-02-13T02:20:57.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:57 trial028 ceph-mon[26701]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2026-02-13T02:20:57.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:57 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:57.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:57 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-13T02:20:57.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:57 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-13T02:20:57.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:57 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-13T02:20:57.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:57 trial028 ceph-mon[26701]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2026-02-13T02:20:57.996 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:57 trial028 ceph-mon[26701]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2026-02-13T02:20:57.996 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:57 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:57.996 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:57 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:57.996 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:57 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:57.996 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:57 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:57.996 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:57 trial028 ceph-mon[26701]: pgmap v50: 1 pgs: 1 creating+peering; 0 B data, 79 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T02:20:58.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:57 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2026-02-13T02:20:58.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:57 trial107 ceph-mon[27422]: osdmap e15: 4 total, 3 up, 4 in 2026-02-13T02:20:58.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:57 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-13T02:20:58.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:57 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:58.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:57 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:58.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:57 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:58.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:57 trial107 ceph-mon[27422]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2026-02-13T02:20:58.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:57 trial107 ceph-mon[27422]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2026-02-13T02:20:58.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:57 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-13T02:20:58.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:57 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-13T02:20:58.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:57 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-13T02:20:58.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:57 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:58.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:57 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-13T02:20:58.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:57 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-13T02:20:58.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:57 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-13T02:20:58.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:57 trial107 ceph-mon[27422]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2026-02-13T02:20:58.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:57 trial107 ceph-mon[27422]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2026-02-13T02:20:58.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:57 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:58.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:57 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-13T02:20:58.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:57 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-13T02:20:58.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:57 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-13T02:20:58.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:57 trial107 ceph-mon[27422]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2026-02-13T02:20:58.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:57 trial107 ceph-mon[27422]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2026-02-13T02:20:58.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:57 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:58.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:57 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:58.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:57 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:58.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:57 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:58.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:57 trial107 ceph-mon[27422]: pgmap v50: 1 pgs: 1 creating+peering; 0 B data, 79 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T02:20:58.140 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:58.137+0000 7fd9827fc6c0 1 -- 10.20.193.27:0/2761513532 <== mgr.14116 v2:10.20.193.27:6800/1837452956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7fd99404ca20 con 0x7fd9700b55f0 2026-02-13T02:20:58.140 INFO:teuthology.orchestra.run.trial027.stdout:Created osd(s) 3 on host 'trial027' 2026-02-13T02:20:58.143 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:58.140+0000 7fd99b7f76c0 1 -- 10.20.193.27:0/2761513532 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd9700b55f0 msgr2=0x7fd9700d5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:58.143 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:58.140+0000 7fd99b7f76c0 1 --2- 10.20.193.27:0/2761513532 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd9700b55f0 0x7fd9700d5aa0 secure :-1 s=READY pgs=90 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd988004750 tx=0x7fd988009260 comp rx=0 tx=0).stop 2026-02-13T02:20:58.143 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:58.140+0000 7fd99b7f76c0 1 -- 10.20.193.27:0/2761513532 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd99410eec0 msgr2=0x7fd9940f8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:58.143 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:58.140+0000 7fd99b7f76c0 1 --2- 10.20.193.27:0/2761513532 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd99410eec0 0x7fd9940f8630 secure :-1 s=READY pgs=151 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd9840092e0 tx=0x7fd984004670 comp rx=0 tx=0).stop 2026-02-13T02:20:58.143 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:58.140+0000 7fd99b7f76c0 1 -- 10.20.193.27:0/2761513532 shutdown_connections 2026-02-13T02:20:58.143 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:58.140+0000 7fd99b7f76c0 1 --2- 10.20.193.27:0/2761513532 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd9700b55f0 0x7fd9700d5aa0 unknown :-1 s=CLOSED pgs=90 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:58.143 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:58.140+0000 7fd99b7f76c0 1 --2- 10.20.193.27:0/2761513532 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd9941109f0 0x7fd9940f8e90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:58.143 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:58.140+0000 7fd99b7f76c0 1 --2- 10.20.193.27:0/2761513532 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd994110040 0x7fd994171380 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:58.144 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:58.140+0000 7fd99b7f76c0 1 --2- 10.20.193.27:0/2761513532 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd99410eec0 0x7fd9940f8630 unknown :-1 s=CLOSED pgs=151 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:58.144 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:58.140+0000 7fd99b7f76c0 1 -- 10.20.193.27:0/2761513532 >> 10.20.193.27:0/2761513532 conn(0x7fd9940ec5c0 msgr2=0x7fd9940eebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:58.144 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:58.140+0000 7fd99b7f76c0 1 -- 10.20.193.27:0/2761513532 shutdown_connections 2026-02-13T02:20:58.144 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:20:58.140+0000 7fd99b7f76c0 1 -- 10.20.193.27:0/2761513532 wait complete. 2026-02-13T02:20:58.185 DEBUG:teuthology.orchestra.run.trial027:osd.3> sudo journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.3.service 2026-02-13T02:20:58.188 INFO:tasks.cephadm:Deploying osd.4 on trial028 with /dev/vg_nvme/lv_3... 2026-02-13T02:20:58.188 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_3 2026-02-13T02:20:58.313 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.b/config 2026-02-13T02:20:58.719 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:58 trial028 ceph-mon[26701]: osdmap e16: 4 total, 3 up, 4 in 2026-02-13T02:20:58.719 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-13T02:20:58.720 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:58.720 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:58.720 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:58.720 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:58.720 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:58.720 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:58 trial028 ceph-mon[26701]: pgmap v52: 1 pgs: 1 creating+peering; 0 B data, 79 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T02:20:58.720 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:58.720 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:58.721 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:58.721 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:58 trial028 ceph-mon[26701]: from='osd.3 [v2:10.20.193.27:6810/382071678,v1:10.20.193.27:6811/382071678]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2026-02-13T02:20:58.721 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:58 trial028 ceph-mon[26701]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:20:58.721 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:58 trial028 ceph-mon[26701]: Cluster is now healthy 2026-02-13T02:20:58.721 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:58.721 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:58.721 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:58 trial028 ceph-mon[26701]: pgmap v53: 1 pgs: 1 creating+peering; 0 B data, 79 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T02:20:58.721 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:58.758 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:20:58.775 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph orch daemon add osd trial028:vg_nvme/lv_3 --skip-validation 2026-02-13T02:20:58.893 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.b/config 2026-02-13T02:20:58.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:58 trial027 ceph-mon[24567]: osdmap e16: 4 total, 3 up, 4 in 2026-02-13T02:20:58.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-13T02:20:58.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:58.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:58.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:58.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:58.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:58.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:58 trial027 ceph-mon[24567]: pgmap v52: 1 pgs: 1 creating+peering; 0 B data, 79 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T02:20:58.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:58.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:58.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:58.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:58 trial027 ceph-mon[24567]: from='osd.3 [v2:10.20.193.27:6810/382071678,v1:10.20.193.27:6811/382071678]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2026-02-13T02:20:58.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:58 trial027 ceph-mon[24567]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:20:58.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:58 trial027 ceph-mon[24567]: Cluster is now healthy 2026-02-13T02:20:58.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:58.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:58.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:58 trial027 ceph-mon[24567]: pgmap v53: 1 pgs: 1 creating+peering; 0 B data, 79 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T02:20:58.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:59.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:58 trial107 ceph-mon[27422]: osdmap e16: 4 total, 3 up, 4 in 2026-02-13T02:20:59.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-13T02:20:59.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:59.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:59.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:59.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:59.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:59.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:58 trial107 ceph-mon[27422]: pgmap v52: 1 pgs: 1 creating+peering; 0 B data, 79 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T02:20:59.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:59.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:59.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:59.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:58 trial107 ceph-mon[27422]: from='osd.3 [v2:10.20.193.27:6810/382071678,v1:10.20.193.27:6811/382071678]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2026-02-13T02:20:59.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:58 trial107 ceph-mon[27422]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:20:59.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:58 trial107 ceph-mon[27422]: Cluster is now healthy 2026-02-13T02:20:59.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:59.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:59.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:58 trial107 ceph-mon[27422]: pgmap v53: 1 pgs: 1 creating+peering; 0 B data, 79 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T02:20:59.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:20:59.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.022+0000 7fe91de1c6c0 1 -- 10.20.193.28:0/2942624873 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe918110a00 msgr2=0x7fe918171390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:59.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.022+0000 7fe91de1c6c0 1 --2- 10.20.193.28:0/2942624873 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe918110a00 0x7fe918171390 secure :-1 s=READY pgs=15 gs=1 cs=0 l=1 c_cookie=5732740523827126792 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe90c00aa40 tx=0x7fe90c02fde0 comp rx=0 tx=0).stop 2026-02-13T02:20:59.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.022+0000 7fe91de1c6c0 1 -- 10.20.193.28:0/2942624873 shutdown_connections 2026-02-13T02:20:59.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.022+0000 7fe91de1c6c0 1 --2- 10.20.193.28:0/2942624873 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe918110a00 0x7fe918171390 unknown :-1 s=CLOSED pgs=15 gs=1 cs=0 l=1 c_cookie=5732740523827126792 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:59.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.022+0000 7fe91de1c6c0 1 --2- 10.20.193.28:0/2942624873 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe918110050 0x7fe9181104c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:59.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.022+0000 7fe91de1c6c0 1 --2- 10.20.193.28:0/2942624873 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe91810eed0 0x7fe91810f2c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:59.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.022+0000 7fe91de1c6c0 1 -- 10.20.193.28:0/2942624873 >> 10.20.193.28:0/2942624873 conn(0x7fe9180ec5f0 msgr2=0x7fe91810ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:20:59.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.022+0000 7fe91de1c6c0 1 -- 10.20.193.28:0/2942624873 shutdown_connections 2026-02-13T02:20:59.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.022+0000 7fe91de1c6c0 1 -- 10.20.193.28:0/2942624873 wait complete. 2026-02-13T02:20:59.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.023+0000 7fe91de1c6c0 1 Processor -- start 2026-02-13T02:20:59.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.023+0000 7fe91de1c6c0 1 -- start start 2026-02-13T02:20:59.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.023+0000 7fe91de1c6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe91810eed0 0x7fe9180f85c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:59.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.023+0000 7fe91de1c6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe918110050 0x7fe918171390 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:59.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.023+0000 7fe91de1c6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe918110a00 0x7fe9180f8e20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:59.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.023+0000 7fe91de1c6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe9180f99c0 con 0x7fe91810eed0 2026-02-13T02:20:59.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.023+0000 7fe91de1c6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe9180f9b30 con 0x7fe918110050 2026-02-13T02:20:59.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.023+0000 7fe91de1c6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe9180f9cf0 con 0x7fe918110a00 2026-02-13T02:20:59.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.023+0000 7fe916ffd6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe918110050 0x7fe918171390 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:59.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.023+0000 7fe916ffd6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe918110050 0x7fe918171390 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:32864/0 (socket says 10.20.193.28:32864) 2026-02-13T02:20:59.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.023+0000 7fe916ffd6c0 1 -- 10.20.193.28:0/3664102124 learned_addr learned my addr 10.20.193.28:0/3664102124 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-13T02:20:59.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.023+0000 7fe917fff6c0 1 --2- 10.20.193.28:0/3664102124 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe918110a00 0x7fe9180f8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:59.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.023+0000 7fe916ffd6c0 1 -- 10.20.193.28:0/3664102124 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe918110a00 msgr2=0x7fe9180f8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:59.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.023+0000 7fe916ffd6c0 1 --2- 10.20.193.28:0/3664102124 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe918110a00 0x7fe9180f8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:59.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.023+0000 7fe916ffd6c0 1 -- 10.20.193.28:0/3664102124 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe91810eed0 msgr2=0x7fe9180f85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:20:59.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.023+0000 7fe916ffd6c0 1 --2- 10.20.193.28:0/3664102124 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe91810eed0 0x7fe9180f85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:20:59.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.023+0000 7fe916ffd6c0 1 -- 10.20.193.28:0/3664102124 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe90c00a660 con 0x7fe918110050 2026-02-13T02:20:59.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.023+0000 7fe917fff6c0 1 --2- 10.20.193.28:0/3664102124 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe918110a00 0x7fe9180f8e20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:20:59.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.023+0000 7fe916ffd6c0 1 --2- 10.20.193.28:0/3664102124 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe918110050 0x7fe918171390 secure :-1 s=READY pgs=16 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe90800b6c0 tx=0x7fe90800bb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:59.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.023+0000 7fe914ff96c0 1 -- 10.20.193.28:0/3664102124 <== mon.2 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe9080044a0 con 0x7fe918110050 2026-02-13T02:20:59.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.023+0000 7fe91de1c6c0 1 -- 10.20.193.28:0/3664102124 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9180f9fd0 con 0x7fe918110050 2026-02-13T02:20:59.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.023+0000 7fe914ff96c0 1 -- 10.20.193.28:0/3664102124 <== mon.2 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fe90800be50 con 0x7fe918110050 2026-02-13T02:20:59.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.023+0000 7fe914ff96c0 1 -- 10.20.193.28:0/3664102124 <== mon.2 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe90800cd00 con 0x7fe918110050 2026-02-13T02:20:59.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.023+0000 7fe91de1c6c0 1 -- 10.20.193.28:0/3664102124 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9180fa4b0 con 0x7fe918110050 2026-02-13T02:20:59.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.024+0000 7fe91de1c6c0 1 -- 10.20.193.28:0/3664102124 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9181752d0 con 0x7fe918110050 2026-02-13T02:20:59.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.025+0000 7fe914ff96c0 1 -- 10.20.193.28:0/3664102124 <== mon.2 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 10) ==== 157010+0+0 (secure 0 0 0) 0x7fe9080026b0 con 0x7fe918110050 2026-02-13T02:20:59.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.026+0000 7fe914ff96c0 1 --2- 10.20.193.28:0/3664102124 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe8ec0b5570 0x7fe8ec0d5a20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:20:59.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.026+0000 7fe914ff96c0 1 -- 10.20.193.28:0/3664102124 <== mon.2 v2:10.20.193.28:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 3150+0+0 (secure 0 0 0) 0x7fe9080e2500 con 0x7fe918110050 2026-02-13T02:20:59.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.026+0000 7fe9177fe6c0 1 --2- 10.20.193.28:0/3664102124 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe8ec0b5570 0x7fe8ec0d5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:20:59.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.026+0000 7fe9177fe6c0 1 --2- 10.20.193.28:0/3664102124 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe8ec0b5570 0x7fe8ec0d5a20 secure :-1 s=READY pgs=96 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe904004750 tx=0x7fe904005d30 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:20:59.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.026+0000 7fe914ff96c0 1 -- 10.20.193.28:0/3664102124 <== mon.2 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fe9080a6400 con 0x7fe918110050 2026-02-13T02:20:59.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.127+0000 7fe91de1c6c0 1 -- 10.20.193.28:0/3664102124 --> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_3", "skip_validation": true, "target": ["mon-mgr", ""]}) -- 0x7fe9180ef240 con 0x7fe8ec0b5570 2026-02-13T02:20:59.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:20:59.146+0000 7fe914ff96c0 1 -- 10.20.193.28:0/3664102124 <== mon.2 v2:10.20.193.28:3300/0 7 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7fe9080a6020 con 0x7fe918110050 2026-02-13T02:20:59.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:59 trial028 ceph-mon[26701]: from='osd.3 [v2:10.20.193.27:6810/382071678,v1:10.20.193.27:6811/382071678]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2026-02-13T02:20:59.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:59 trial028 ceph-mon[26701]: osdmap e17: 4 total, 3 up, 4 in 2026-02-13T02:20:59.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:59 trial028 ceph-mon[26701]: from='osd.3 [v2:10.20.193.27:6810/382071678,v1:10.20.193.27:6811/382071678]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.6836, "args": ["host=trial027", "root=default"]} : dispatch 2026-02-13T02:20:59.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:59 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-13T02:20:59.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:59 trial028 ceph-mon[26701]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_3", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:20:59.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:59 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:59.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:59 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:59.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:59 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:59.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:59 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:59.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:59 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:59.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:59 trial028 ceph-mon[26701]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:20:59.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:59 trial028 ceph-mon[26701]: mgrmap e11: x(active, since 62s), standbys: z, y 2026-02-13T02:20:59.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:59 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:59.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:59 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:59.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:59 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2026-02-13T02:20:59.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:59 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2026-02-13T02:20:59.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:59 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:59.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:59 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:20:59.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:59 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:20:59.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:59 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:20:59.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:59 trial028 ceph-mon[26701]: pgmap v55: 1 pgs: 1 creating+peering; 0 B data, 79 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T02:20:59.746 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:59 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:20:59.746 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:59 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:20:59.746 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:20:59 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:00.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:59 trial107 ceph-mon[27422]: from='osd.3 [v2:10.20.193.27:6810/382071678,v1:10.20.193.27:6811/382071678]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2026-02-13T02:21:00.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:59 trial107 ceph-mon[27422]: osdmap e17: 4 total, 3 up, 4 in 2026-02-13T02:21:00.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:59 trial107 ceph-mon[27422]: from='osd.3 [v2:10.20.193.27:6810/382071678,v1:10.20.193.27:6811/382071678]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.6836, "args": ["host=trial027", "root=default"]} : dispatch 2026-02-13T02:21:00.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:59 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-13T02:21:00.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:59 trial107 ceph-mon[27422]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_3", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:21:00.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:59 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:00.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:59 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:00.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:59 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:00.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:59 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:00.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:59 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:00.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:59 trial107 ceph-mon[27422]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:00.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:59 trial107 ceph-mon[27422]: mgrmap e11: x(active, since 62s), standbys: z, y 2026-02-13T02:21:00.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:59 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:00.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:59 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:00.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:59 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:00.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:59 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:00.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:59 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:00.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:59 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:00.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:59 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:00.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:59 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:00.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:59 trial107 ceph-mon[27422]: pgmap v55: 1 pgs: 1 creating+peering; 0 B data, 79 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T02:21:00.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:59 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:00.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:59 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:00.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:20:59 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:00.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:59 trial027 ceph-mon[24567]: from='osd.3 [v2:10.20.193.27:6810/382071678,v1:10.20.193.27:6811/382071678]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2026-02-13T02:21:00.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:59 trial027 ceph-mon[24567]: osdmap e17: 4 total, 3 up, 4 in 2026-02-13T02:21:00.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:59 trial027 ceph-mon[24567]: from='osd.3 [v2:10.20.193.27:6810/382071678,v1:10.20.193.27:6811/382071678]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.6836, "args": ["host=trial027", "root=default"]} : dispatch 2026-02-13T02:21:00.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:59 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-13T02:21:00.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:59 trial027 ceph-mon[24567]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_3", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:21:00.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:59 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:00.160 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:59 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:00.160 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:59 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:00.160 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:59 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:00.160 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:59 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:00.160 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:59 trial027 ceph-mon[24567]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:00.160 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:59 trial027 ceph-mon[24567]: mgrmap e11: x(active, since 62s), standbys: z, y 2026-02-13T02:21:00.160 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:59 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:00.160 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:59 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:00.160 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:59 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:00.160 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:59 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:00.161 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:59 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:00.161 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:59 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:00.161 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:59 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:00.161 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:59 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:00.161 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:59 trial027 ceph-mon[24567]: pgmap v55: 1 pgs: 1 creating+peering; 0 B data, 79 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-13T02:21:00.161 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:59 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:00.161 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:59 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:00.161 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:20:59 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:00.162 INFO:journalctl@ceph.osd.3.trial027.stdout:Feb 13 02:20:59 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-3[43064]: 2026-02-13T02:20:59.665+0000 7fcb806e36c0 -1 osd.3 0 waiting for initial osdmap 2026-02-13T02:21:00.162 INFO:journalctl@ceph.osd.3.trial027.stdout:Feb 13 02:20:59 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-3[43064]: 2026-02-13T02:20:59.670+0000 7fcb7c5a76c0 -1 osd.3 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-13T02:21:00.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:00 trial028 ceph-mon[26701]: from='osd.3 [v2:10.20.193.27:6810/382071678,v1:10.20.193.27:6811/382071678]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.6836, "args": ["host=trial027", "root=default"]}]': finished 2026-02-13T02:21:00.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:00 trial028 ceph-mon[26701]: osdmap e18: 4 total, 3 up, 4 in 2026-02-13T02:21:00.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:00 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-13T02:21:00.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:00 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-13T02:21:00.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:00 trial028 ceph-mon[26701]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4beaec0d-7b39-4bb0-9770-3dc4c848b106"} : dispatch 2026-02-13T02:21:00.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:00 trial028 ceph-mon[26701]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4beaec0d-7b39-4bb0-9770-3dc4c848b106"}]': finished 2026-02-13T02:21:00.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:00 trial028 ceph-mon[26701]: osd.3 [v2:10.20.193.27:6810/382071678,v1:10.20.193.27:6811/382071678] boot 2026-02-13T02:21:00.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:00 trial028 ceph-mon[26701]: osdmap e19: 5 total, 4 up, 5 in 2026-02-13T02:21:00.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:00 trial028 ceph-mon[26701]: from='client.? 10.20.193.28:0/699523447' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4beaec0d-7b39-4bb0-9770-3dc4c848b106"} : dispatch 2026-02-13T02:21:00.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:00 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-13T02:21:00.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:00 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-13T02:21:00.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:00 trial028 ceph-mon[26701]: from='client.? 10.20.193.28:0/260776692' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:21:00.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:00 trial028 ceph-mon[26701]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:00.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:00 trial028 ceph-mon[26701]: Cluster is now healthy 2026-02-13T02:21:01.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:00 trial107 ceph-mon[27422]: from='osd.3 [v2:10.20.193.27:6810/382071678,v1:10.20.193.27:6811/382071678]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.6836, "args": ["host=trial027", "root=default"]}]': finished 2026-02-13T02:21:01.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:00 trial107 ceph-mon[27422]: osdmap e18: 4 total, 3 up, 4 in 2026-02-13T02:21:01.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:00 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-13T02:21:01.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:00 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-13T02:21:01.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:00 trial107 ceph-mon[27422]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4beaec0d-7b39-4bb0-9770-3dc4c848b106"} : dispatch 2026-02-13T02:21:01.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:00 trial107 ceph-mon[27422]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4beaec0d-7b39-4bb0-9770-3dc4c848b106"}]': finished 2026-02-13T02:21:01.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:00 trial107 ceph-mon[27422]: osd.3 [v2:10.20.193.27:6810/382071678,v1:10.20.193.27:6811/382071678] boot 2026-02-13T02:21:01.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:00 trial107 ceph-mon[27422]: osdmap e19: 5 total, 4 up, 5 in 2026-02-13T02:21:01.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:00 trial107 ceph-mon[27422]: from='client.? 10.20.193.28:0/699523447' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4beaec0d-7b39-4bb0-9770-3dc4c848b106"} : dispatch 2026-02-13T02:21:01.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:00 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-13T02:21:01.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:00 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-13T02:21:01.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:00 trial107 ceph-mon[27422]: from='client.? 10.20.193.28:0/260776692' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:21:01.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:00 trial107 ceph-mon[27422]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:01.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:00 trial107 ceph-mon[27422]: Cluster is now healthy 2026-02-13T02:21:01.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:00 trial027 ceph-mon[24567]: from='osd.3 [v2:10.20.193.27:6810/382071678,v1:10.20.193.27:6811/382071678]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.6836, "args": ["host=trial027", "root=default"]}]': finished 2026-02-13T02:21:01.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:00 trial027 ceph-mon[24567]: osdmap e18: 4 total, 3 up, 4 in 2026-02-13T02:21:01.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:00 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-13T02:21:01.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:00 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-13T02:21:01.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:00 trial027 ceph-mon[24567]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4beaec0d-7b39-4bb0-9770-3dc4c848b106"} : dispatch 2026-02-13T02:21:01.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:00 trial027 ceph-mon[24567]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4beaec0d-7b39-4bb0-9770-3dc4c848b106"}]': finished 2026-02-13T02:21:01.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:00 trial027 ceph-mon[24567]: osd.3 [v2:10.20.193.27:6810/382071678,v1:10.20.193.27:6811/382071678] boot 2026-02-13T02:21:01.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:00 trial027 ceph-mon[24567]: osdmap e19: 5 total, 4 up, 5 in 2026-02-13T02:21:01.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:00 trial027 ceph-mon[24567]: from='client.? 10.20.193.28:0/699523447' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4beaec0d-7b39-4bb0-9770-3dc4c848b106"} : dispatch 2026-02-13T02:21:01.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:00 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-13T02:21:01.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:00 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-13T02:21:01.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:00 trial027 ceph-mon[24567]: from='client.? 10.20.193.28:0/260776692' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:21:01.160 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:00 trial027 ceph-mon[24567]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:01.160 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:00 trial027 ceph-mon[24567]: Cluster is now healthy 2026-02-13T02:21:02.163 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:01 trial028 ceph-mon[26701]: purged_snaps scrub starts 2026-02-13T02:21:02.163 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:01 trial028 ceph-mon[26701]: purged_snaps scrub ok 2026-02-13T02:21:02.163 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:01 trial028 ceph-mon[26701]: osdmap e20: 5 total, 4 up, 5 in 2026-02-13T02:21:02.163 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:01 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-13T02:21:02.163 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:01 trial028 ceph-mon[26701]: pgmap v59: 1 pgs: 1 creating+peering; 0 B data, 106 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-13T02:21:02.163 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:01 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:02.163 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:01 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:02.163 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:01 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:02.163 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:01 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:02.164 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:01 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:02.164 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:01 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:02.164 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:01 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:02.164 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:01 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:02.164 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:01 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:02.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:01 trial107 ceph-mon[27422]: purged_snaps scrub starts 2026-02-13T02:21:02.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:01 trial107 ceph-mon[27422]: purged_snaps scrub ok 2026-02-13T02:21:02.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:01 trial107 ceph-mon[27422]: osdmap e20: 5 total, 4 up, 5 in 2026-02-13T02:21:02.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:01 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-13T02:21:02.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:01 trial107 ceph-mon[27422]: pgmap v59: 1 pgs: 1 creating+peering; 0 B data, 106 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-13T02:21:02.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:01 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:02.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:01 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:02.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:01 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:02.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:01 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:02.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:01 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:02.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:01 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:02.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:01 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:02.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:01 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:02.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:01 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:02.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:01 trial027 ceph-mon[24567]: purged_snaps scrub starts 2026-02-13T02:21:02.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:01 trial027 ceph-mon[24567]: purged_snaps scrub ok 2026-02-13T02:21:02.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:01 trial027 ceph-mon[24567]: osdmap e20: 5 total, 4 up, 5 in 2026-02-13T02:21:02.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:01 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-13T02:21:02.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:01 trial027 ceph-mon[24567]: pgmap v59: 1 pgs: 1 creating+peering; 0 B data, 106 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-13T02:21:02.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:01 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:02.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:01 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:02.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:01 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:02.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:01 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:02.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:01 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:02.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:01 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:02.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:01 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:02.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:01 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:02.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:01 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:03.240 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:02 trial028 ceph-mon[26701]: pgmap v60: 1 pgs: 1 remapped+peering; 577 KiB data, 106 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-13T02:21:03.240 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:02 trial028 ceph-mon[26701]: pgmap v61: 1 pgs: 1 remapped+peering; 577 KiB data, 106 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-13T02:21:03.241 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:02 trial028 ceph-mon[26701]: osdmap e21: 5 total, 4 up, 5 in 2026-02-13T02:21:03.241 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:02 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-13T02:21:03.241 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:02 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2026-02-13T02:21:03.241 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:02 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:03.241 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:02 trial028 ceph-mon[26701]: pgmap v63: 1 pgs: 1 remapped+peering; 577 KiB data, 106 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-13T02:21:03.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:02 trial107 ceph-mon[27422]: pgmap v60: 1 pgs: 1 remapped+peering; 577 KiB data, 106 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-13T02:21:03.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:02 trial107 ceph-mon[27422]: pgmap v61: 1 pgs: 1 remapped+peering; 577 KiB data, 106 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-13T02:21:03.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:02 trial107 ceph-mon[27422]: osdmap e21: 5 total, 4 up, 5 in 2026-02-13T02:21:03.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:02 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-13T02:21:03.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:02 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2026-02-13T02:21:03.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:02 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:03.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:02 trial107 ceph-mon[27422]: pgmap v63: 1 pgs: 1 remapped+peering; 577 KiB data, 106 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-13T02:21:03.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:02 trial027 ceph-mon[24567]: pgmap v60: 1 pgs: 1 remapped+peering; 577 KiB data, 106 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-13T02:21:03.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:02 trial027 ceph-mon[24567]: pgmap v61: 1 pgs: 1 remapped+peering; 577 KiB data, 106 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-13T02:21:03.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:02 trial027 ceph-mon[24567]: osdmap e21: 5 total, 4 up, 5 in 2026-02-13T02:21:03.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:02 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-13T02:21:03.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:02 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2026-02-13T02:21:03.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:02 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:03.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:02 trial027 ceph-mon[24567]: pgmap v63: 1 pgs: 1 remapped+peering; 577 KiB data, 106 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-13T02:21:04.203 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:03 trial028 ceph-mon[26701]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:04.203 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:03 trial028 ceph-mon[26701]: osdmap e22: 5 total, 4 up, 5 in 2026-02-13T02:21:04.203 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:03 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-13T02:21:04.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:03 trial107 ceph-mon[27422]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:04.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:03 trial107 ceph-mon[27422]: osdmap e22: 5 total, 4 up, 5 in 2026-02-13T02:21:04.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:03 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-13T02:21:04.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:03 trial027 ceph-mon[24567]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:04.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:03 trial027 ceph-mon[24567]: osdmap e22: 5 total, 4 up, 5 in 2026-02-13T02:21:04.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:03 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-13T02:21:05.182 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:04 trial028 ceph-mon[26701]: osdmap e23: 5 total, 4 up, 5 in 2026-02-13T02:21:05.182 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:04 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-13T02:21:05.182 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:04 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:05.182 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:04 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:05.182 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:04 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:05.182 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:04 trial028 ceph-mon[26701]: pgmap v66: 1 pgs: 1 remapped+peering; 577 KiB data, 106 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-13T02:21:05.182 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:04 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:05.182 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:04 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:05.183 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:04 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:05.183 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:04 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:05.183 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:04 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:05.183 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:04 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:05.183 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:04 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:05.183 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:04 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:05.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:04 trial107 ceph-mon[27422]: osdmap e23: 5 total, 4 up, 5 in 2026-02-13T02:21:05.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:04 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-13T02:21:05.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:04 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:05.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:04 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:05.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:04 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:05.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:04 trial107 ceph-mon[27422]: pgmap v66: 1 pgs: 1 remapped+peering; 577 KiB data, 106 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-13T02:21:05.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:04 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:05.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:04 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:05.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:04 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:05.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:04 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:05.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:04 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:05.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:04 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:05.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:04 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:05.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:04 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:05.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:05.309+0000 7fe914ff96c0 1 -- 10.20.193.28:0/3664102124 <== mgr.14116 v2:10.20.193.27:6800/1837452956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7fe9180ef240 con 0x7fe8ec0b5570 2026-02-13T02:21:05.308 INFO:teuthology.orchestra.run.trial028.stdout:Created osd(s) 4 on host 'trial028' 2026-02-13T02:21:05.310 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:05.311+0000 7fe91de1c6c0 1 -- 10.20.193.28:0/3664102124 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe8ec0b5570 msgr2=0x7fe8ec0d5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:05.310 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:05.311+0000 7fe91de1c6c0 1 --2- 10.20.193.28:0/3664102124 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe8ec0b5570 0x7fe8ec0d5a20 secure :-1 s=READY pgs=96 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe904004750 tx=0x7fe904005d30 comp rx=0 tx=0).stop 2026-02-13T02:21:05.310 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:05.311+0000 7fe91de1c6c0 1 -- 10.20.193.28:0/3664102124 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe918110050 msgr2=0x7fe918171390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:05.310 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:05.311+0000 7fe91de1c6c0 1 --2- 10.20.193.28:0/3664102124 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe918110050 0x7fe918171390 secure :-1 s=READY pgs=16 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe90800b6c0 tx=0x7fe90800bb90 comp rx=0 tx=0).stop 2026-02-13T02:21:05.310 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:05.311+0000 7fe91de1c6c0 1 -- 10.20.193.28:0/3664102124 shutdown_connections 2026-02-13T02:21:05.311 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:05.311+0000 7fe91de1c6c0 1 --2- 10.20.193.28:0/3664102124 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe8ec0b5570 0x7fe8ec0d5a20 unknown :-1 s=CLOSED pgs=96 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:05.311 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:05.311+0000 7fe91de1c6c0 1 --2- 10.20.193.28:0/3664102124 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe918110a00 0x7fe9180f8e20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:05.311 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:05.311+0000 7fe91de1c6c0 1 --2- 10.20.193.28:0/3664102124 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe918110050 0x7fe918171390 unknown :-1 s=CLOSED pgs=16 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:05.311 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:05.311+0000 7fe91de1c6c0 1 --2- 10.20.193.28:0/3664102124 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe91810eed0 0x7fe9180f85c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:05.311 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:05.311+0000 7fe91de1c6c0 1 -- 10.20.193.28:0/3664102124 >> 10.20.193.28:0/3664102124 conn(0x7fe9180ec5f0 msgr2=0x7fe9180eeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:21:05.311 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:05.311+0000 7fe91de1c6c0 1 -- 10.20.193.28:0/3664102124 shutdown_connections 2026-02-13T02:21:05.311 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:05.311+0000 7fe91de1c6c0 1 -- 10.20.193.28:0/3664102124 wait complete. 2026-02-13T02:21:05.352 DEBUG:teuthology.orchestra.run.trial028:osd.4> sudo journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.4.service 2026-02-13T02:21:05.354 INFO:tasks.cephadm:Deploying osd.5 on trial107 with /dev/vg_nvme/lv_3... 2026-02-13T02:21:05.354 DEBUG:teuthology.orchestra.run.trial107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_3 2026-02-13T02:21:05.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:04 trial027 ceph-mon[24567]: osdmap e23: 5 total, 4 up, 5 in 2026-02-13T02:21:05.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:04 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-13T02:21:05.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:04 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:05.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:04 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:05.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:04 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:05.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:04 trial027 ceph-mon[24567]: pgmap v66: 1 pgs: 1 remapped+peering; 577 KiB data, 106 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-13T02:21:05.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:04 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:05.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:04 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:05.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:04 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:05.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:04 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:05.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:04 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:05.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:04 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:05.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:04 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:05.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:04 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:05.480 INFO:teuthology.orchestra.run.trial107.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.c/config 2026-02-13T02:21:05.920 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:21:05.937 DEBUG:teuthology.orchestra.run.trial107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph orch daemon add osd trial107:vg_nvme/lv_3 --skip-validation 2026-02-13T02:21:06.058 INFO:teuthology.orchestra.run.trial107.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.c/config 2026-02-13T02:21:06.173 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.175+0000 7f30b73f46c0 1 -- 10.20.193.107:0/1628025282 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f30b010ddd0 msgr2=0x7f30b010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:06.174 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.175+0000 7f30b73f46c0 1 --2- 10.20.193.107:0/1628025282 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f30b010ddd0 0x7f30b010e240 secure :-1 s=READY pgs=17 gs=1 cs=0 l=1 c_cookie=5517924640445079620 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f30a4009970 tx=0x7f30a402ed50 comp rx=0 tx=0).stop 2026-02-13T02:21:06.174 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:05 trial107 ceph-mon[27422]: pgmap v67: 1 pgs: 1 remapped+peering; 577 KiB data, 106 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-13T02:21:06.174 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:05 trial107 ceph-mon[27422]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2026-02-13T02:21:06.174 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:05 trial107 ceph-mon[27422]: from='osd.4 [v2:10.20.193.28:6808/711804032,v1:10.20.193.28:6809/711804032]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2026-02-13T02:21:06.174 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:05 trial107 ceph-mon[27422]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:06.175 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:05 trial107 ceph-mon[27422]: Cluster is now healthy 2026-02-13T02:21:06.175 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:05 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:06.175 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:05 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:06.175 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:05 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:06.178 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.175+0000 7f30b73f46c0 1 -- 10.20.193.107:0/1628025282 shutdown_connections 2026-02-13T02:21:06.178 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.175+0000 7f30b73f46c0 1 --2- 10.20.193.107:0/1628025282 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f30b010e780 0x7f30b016f110 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:06.178 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.175+0000 7f30b73f46c0 1 --2- 10.20.193.107:0/1628025282 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f30b010ddd0 0x7f30b010e240 unknown :-1 s=CLOSED pgs=17 gs=1 cs=0 l=1 c_cookie=5517924640445079620 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:06.178 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.175+0000 7f30b73f46c0 1 --2- 10.20.193.107:0/1628025282 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f30b010cc50 0x7f30b010d040 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:06.178 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.175+0000 7f30b73f46c0 1 -- 10.20.193.107:0/1628025282 >> 10.20.193.107:0/1628025282 conn(0x7f30b00ea370 msgr2=0x7f30b010a780 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:21:06.178 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.175+0000 7f30b73f46c0 1 -- 10.20.193.107:0/1628025282 shutdown_connections 2026-02-13T02:21:06.178 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.176+0000 7f30b73f46c0 1 -- 10.20.193.107:0/1628025282 wait complete. 2026-02-13T02:21:06.178 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.176+0000 7f30b73f46c0 1 Processor -- start 2026-02-13T02:21:06.178 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.176+0000 7f30b73f46c0 1 -- start start 2026-02-13T02:21:06.179 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.176+0000 7f30b73f46c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f30b010cc50 0x7f30b00f8640 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:06.179 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.176+0000 7f30b73f46c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f30b010ddd0 0x7f30b016f110 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:06.179 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.176+0000 7f30b73f46c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f30b010e780 0x7f30b00f8ea0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:06.179 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.176+0000 7f30b73f46c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f30b00f9a40 con 0x7f30b010cc50 2026-02-13T02:21:06.179 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.176+0000 7f30b73f46c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f30b00f9bb0 con 0x7f30b010ddd0 2026-02-13T02:21:06.179 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.176+0000 7f30b73f46c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f30b00f9d70 con 0x7f30b010e780 2026-02-13T02:21:06.179 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.176+0000 7f30b596a6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f30b010e780 0x7f30b00f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:06.179 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.176+0000 7f30b596a6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f30b010e780 0x7f30b00f8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.107:3300/0 says I am v2:10.20.193.107:57886/0 (socket says 10.20.193.107:57886) 2026-02-13T02:21:06.179 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.176+0000 7f30b596a6c0 1 -- 10.20.193.107:0/4023313429 learned_addr learned my addr 10.20.193.107:0/4023313429 (peer_addr_for_me v2:10.20.193.107:0/0) 2026-02-13T02:21:06.179 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.176+0000 7f30b51696c0 1 --2- 10.20.193.107:0/4023313429 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f30b010cc50 0x7f30b00f8640 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:06.179 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.176+0000 7f30b49686c0 1 --2- 10.20.193.107:0/4023313429 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f30b010ddd0 0x7f30b016f110 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:06.180 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.176+0000 7f30b596a6c0 1 -- 10.20.193.107:0/4023313429 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f30b010ddd0 msgr2=0x7f30b016f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:06.180 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.176+0000 7f30b596a6c0 1 --2- 10.20.193.107:0/4023313429 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f30b010ddd0 0x7f30b016f110 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:06.180 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.176+0000 7f30b596a6c0 1 -- 10.20.193.107:0/4023313429 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f30b010cc50 msgr2=0x7f30b00f8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:06.180 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.176+0000 7f30b596a6c0 1 --2- 10.20.193.107:0/4023313429 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f30b010cc50 0x7f30b00f8640 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:06.180 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.176+0000 7f30b596a6c0 1 -- 10.20.193.107:0/4023313429 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a4009590 con 0x7f30b010e780 2026-02-13T02:21:06.180 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.176+0000 7f30b51696c0 1 --2- 10.20.193.107:0/4023313429 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f30b010cc50 0x7f30b00f8640 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:21:06.180 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.176+0000 7f30b596a6c0 1 --2- 10.20.193.107:0/4023313429 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f30b010e780 0x7f30b00f8ea0 secure :-1 s=READY pgs=18 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f30ac00c6c0 tx=0x7f30ac00cb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:21:06.180 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.176+0000 7f309e7fc6c0 1 -- 10.20.193.107:0/4023313429 <== mon.1 v2:10.20.193.107:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f30ac003c40 con 0x7f30b010e780 2026-02-13T02:21:06.180 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.177+0000 7f309e7fc6c0 1 -- 10.20.193.107:0/4023313429 <== mon.1 v2:10.20.193.107:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f30ac00a030 con 0x7f30b010e780 2026-02-13T02:21:06.180 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.177+0000 7f30b73f46c0 1 -- 10.20.193.107:0/4023313429 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30b00fa050 con 0x7f30b010e780 2026-02-13T02:21:06.180 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.177+0000 7f309e7fc6c0 1 -- 10.20.193.107:0/4023313429 <== mon.1 v2:10.20.193.107:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f30ac00db70 con 0x7f30b010e780 2026-02-13T02:21:06.180 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.177+0000 7f30b73f46c0 1 -- 10.20.193.107:0/4023313429 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30b00fa530 con 0x7f30b010e780 2026-02-13T02:21:06.181 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.177+0000 7f30b73f46c0 1 -- 10.20.193.107:0/4023313429 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30b0173020 con 0x7f30b010e780 2026-02-13T02:21:06.181 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.178+0000 7f309e7fc6c0 1 -- 10.20.193.107:0/4023313429 <== mon.1 v2:10.20.193.107:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7f30ac00dd20 con 0x7f30b010e780 2026-02-13T02:21:06.181 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.179+0000 7f309e7fc6c0 1 --2- 10.20.193.107:0/4023313429 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f308c0b5600 0x7f308c0d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:06.181 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.179+0000 7f309e7fc6c0 1 -- 10.20.193.107:0/4023313429 <== mon.1 v2:10.20.193.107:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 3582+0+0 (secure 0 0 0) 0x7f30ac0e37c0 con 0x7f30b010e780 2026-02-13T02:21:06.181 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.179+0000 7f30b51696c0 1 --2- 10.20.193.107:0/4023313429 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f308c0b5600 0x7f308c0d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:06.181 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.179+0000 7f309e7fc6c0 1 -- 10.20.193.107:0/4023313429 <== mon.1 v2:10.20.193.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f30ac0a73a0 con 0x7f30b010e780 2026-02-13T02:21:06.181 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.180+0000 7f30b51696c0 1 --2- 10.20.193.107:0/4023313429 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f308c0b5600 0x7f308c0d5ab0 secure :-1 s=READY pgs=101 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f30a0004750 tx=0x7f30a00095b0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:21:06.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:05 trial028 ceph-mon[26701]: pgmap v67: 1 pgs: 1 remapped+peering; 577 KiB data, 106 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-13T02:21:06.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:05 trial028 ceph-mon[26701]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2026-02-13T02:21:06.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:05 trial028 ceph-mon[26701]: from='osd.4 [v2:10.20.193.28:6808/711804032,v1:10.20.193.28:6809/711804032]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2026-02-13T02:21:06.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:05 trial028 ceph-mon[26701]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:06.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:05 trial028 ceph-mon[26701]: Cluster is now healthy 2026-02-13T02:21:06.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:05 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:06.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:05 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:06.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:05 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:06.274 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:06.276+0000 7f30b73f46c0 1 -- 10.20.193.107:0/4023313429 --> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "trial107:vg_nvme/lv_3", "skip_validation": true, "target": ["mon-mgr", ""]}) -- 0x7f30b0061dd0 con 0x7f308c0b5600 2026-02-13T02:21:06.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:05 trial027 ceph-mon[24567]: pgmap v67: 1 pgs: 1 remapped+peering; 577 KiB data, 106 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-13T02:21:06.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:05 trial027 ceph-mon[24567]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2026-02-13T02:21:06.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:05 trial027 ceph-mon[24567]: from='osd.4 [v2:10.20.193.28:6808/711804032,v1:10.20.193.28:6809/711804032]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2026-02-13T02:21:06.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:05 trial027 ceph-mon[24567]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:06.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:05 trial027 ceph-mon[24567]: Cluster is now healthy 2026-02-13T02:21:06.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:05 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:06.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:05 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:06.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:05 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:07.178 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:06 trial107 ceph-mon[27422]: pgmap v68: 1 pgs: 1 remapped+peering; 577 KiB data, 106 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-13T02:21:07.178 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:06 trial107 ceph-mon[27422]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2026-02-13T02:21:07.179 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:06 trial107 ceph-mon[27422]: osdmap e24: 5 total, 4 up, 5 in 2026-02-13T02:21:07.179 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:06 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-13T02:21:07.179 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:06 trial107 ceph-mon[27422]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-13T02:21:07.179 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:06 trial107 ceph-mon[27422]: from='osd.4 [v2:10.20.193.28:6808/711804032,v1:10.20.193.28:6809/711804032]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-13T02:21:07.179 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:06 trial107 ceph-mon[27422]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial107:vg_nvme/lv_3", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:21:07.179 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:06 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:07.179 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:06 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:07.179 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:06 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:07.179 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:06 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:07.179 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:06 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:07.180 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:06 trial107 ceph-mon[27422]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:07.180 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:06 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:07.180 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:06 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:07.180 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:06 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:07.180 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:06 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:07.180 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:06 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:07.180 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:06 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:07.180 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:06 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:07.180 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:06 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:07.180 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:06 trial107 ceph-mon[27422]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 2.7 TiB / 2.7 TiB avail; 165 KiB/s, 0 objects/s recovering 2026-02-13T02:21:07.180 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:06 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:07.181 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:06 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:07.181 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:06 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:07.244 INFO:journalctl@ceph.osd.4.trial028.stdout:Feb 13 02:21:06 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-4[37681]: 2026-02-13T02:21:06.968+0000 7f37e4e276c0 -1 osd.4 0 waiting for initial osdmap 2026-02-13T02:21:07.244 INFO:journalctl@ceph.osd.4.trial028.stdout:Feb 13 02:21:06 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-4[37681]: 2026-02-13T02:21:06.971+0000 7f37e0ceb6c0 -1 osd.4 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-13T02:21:07.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:06 trial028 ceph-mon[26701]: pgmap v68: 1 pgs: 1 remapped+peering; 577 KiB data, 106 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-13T02:21:07.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:06 trial028 ceph-mon[26701]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2026-02-13T02:21:07.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:06 trial028 ceph-mon[26701]: osdmap e24: 5 total, 4 up, 5 in 2026-02-13T02:21:07.245 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:06 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-13T02:21:07.245 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:06 trial028 ceph-mon[26701]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-13T02:21:07.245 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:06 trial028 ceph-mon[26701]: from='osd.4 [v2:10.20.193.28:6808/711804032,v1:10.20.193.28:6809/711804032]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-13T02:21:07.245 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:06 trial028 ceph-mon[26701]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial107:vg_nvme/lv_3", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:21:07.245 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:06 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:07.245 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:06 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:07.245 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:06 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:07.245 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:06 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:07.245 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:06 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:07.245 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:06 trial028 ceph-mon[26701]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:07.245 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:06 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:07.246 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:06 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:07.246 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:06 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:07.246 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:06 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:07.246 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:06 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:07.246 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:06 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:07.246 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:06 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:07.246 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:06 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:07.246 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:06 trial028 ceph-mon[26701]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 2.7 TiB / 2.7 TiB avail; 165 KiB/s, 0 objects/s recovering 2026-02-13T02:21:07.246 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:06 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:07.246 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:06 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:07.246 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:06 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:07.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:06 trial027 ceph-mon[24567]: pgmap v68: 1 pgs: 1 remapped+peering; 577 KiB data, 106 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-13T02:21:07.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:06 trial027 ceph-mon[24567]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2026-02-13T02:21:07.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:06 trial027 ceph-mon[24567]: osdmap e24: 5 total, 4 up, 5 in 2026-02-13T02:21:07.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:06 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-13T02:21:07.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:06 trial027 ceph-mon[24567]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-13T02:21:07.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:06 trial027 ceph-mon[24567]: from='osd.4 [v2:10.20.193.28:6808/711804032,v1:10.20.193.28:6809/711804032]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-13T02:21:07.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:06 trial027 ceph-mon[24567]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial107:vg_nvme/lv_3", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:21:07.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:06 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:07.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:06 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:07.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:06 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:07.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:06 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:07.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:06 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:07.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:06 trial027 ceph-mon[24567]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:07.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:06 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:07.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:06 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:07.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:06 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:07.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:06 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:07.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:06 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:07.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:06 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:07.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:06 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:07.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:06 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:07.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:06 trial027 ceph-mon[24567]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 2.7 TiB / 2.7 TiB avail; 165 KiB/s, 0 objects/s recovering 2026-02-13T02:21:07.411 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:06 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:07.411 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:06 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:07.411 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:06 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:08.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:07 trial028 ceph-mon[26701]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.6836, "args": ["host=trial028", "root=default"]}]': finished 2026-02-13T02:21:08.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:07 trial028 ceph-mon[26701]: osdmap e25: 5 total, 4 up, 5 in 2026-02-13T02:21:08.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:07 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-13T02:21:08.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:07 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-13T02:21:08.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:07 trial028 ceph-mon[26701]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bb16754e-66f2-4951-9f31-d1dde591ec92"} : dispatch 2026-02-13T02:21:08.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:07 trial028 ceph-mon[26701]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb16754e-66f2-4951-9f31-d1dde591ec92"}]': finished 2026-02-13T02:21:08.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:07 trial028 ceph-mon[26701]: osd.4 [v2:10.20.193.28:6808/711804032,v1:10.20.193.28:6809/711804032] boot 2026-02-13T02:21:08.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:07 trial028 ceph-mon[26701]: osdmap e26: 6 total, 5 up, 6 in 2026-02-13T02:21:08.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:07 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-13T02:21:08.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:07 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-13T02:21:08.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:07 trial028 ceph-mon[26701]: from='client.? 10.20.193.107:0/2418694110' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bb16754e-66f2-4951-9f31-d1dde591ec92"} : dispatch 2026-02-13T02:21:08.245 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:07 trial028 ceph-mon[26701]: from='client.? 10.20.193.107:0/1929671483' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:21:08.245 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:07 trial028 ceph-mon[26701]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:08.245 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:07 trial028 ceph-mon[26701]: Cluster is now healthy 2026-02-13T02:21:08.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:07 trial107 ceph-mon[27422]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.6836, "args": ["host=trial028", "root=default"]}]': finished 2026-02-13T02:21:08.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:07 trial107 ceph-mon[27422]: osdmap e25: 5 total, 4 up, 5 in 2026-02-13T02:21:08.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:07 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-13T02:21:08.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:07 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-13T02:21:08.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:07 trial107 ceph-mon[27422]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bb16754e-66f2-4951-9f31-d1dde591ec92"} : dispatch 2026-02-13T02:21:08.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:07 trial107 ceph-mon[27422]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb16754e-66f2-4951-9f31-d1dde591ec92"}]': finished 2026-02-13T02:21:08.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:07 trial107 ceph-mon[27422]: osd.4 [v2:10.20.193.28:6808/711804032,v1:10.20.193.28:6809/711804032] boot 2026-02-13T02:21:08.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:07 trial107 ceph-mon[27422]: osdmap e26: 6 total, 5 up, 6 in 2026-02-13T02:21:08.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:07 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-13T02:21:08.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:07 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-13T02:21:08.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:07 trial107 ceph-mon[27422]: from='client.? 10.20.193.107:0/2418694110' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bb16754e-66f2-4951-9f31-d1dde591ec92"} : dispatch 2026-02-13T02:21:08.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:07 trial107 ceph-mon[27422]: from='client.? 10.20.193.107:0/1929671483' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:21:08.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:07 trial107 ceph-mon[27422]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:08.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:07 trial107 ceph-mon[27422]: Cluster is now healthy 2026-02-13T02:21:08.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:07 trial027 ceph-mon[24567]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.6836, "args": ["host=trial028", "root=default"]}]': finished 2026-02-13T02:21:08.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:07 trial027 ceph-mon[24567]: osdmap e25: 5 total, 4 up, 5 in 2026-02-13T02:21:08.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:07 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-13T02:21:08.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:07 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-13T02:21:08.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:07 trial027 ceph-mon[24567]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bb16754e-66f2-4951-9f31-d1dde591ec92"} : dispatch 2026-02-13T02:21:08.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:07 trial027 ceph-mon[24567]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb16754e-66f2-4951-9f31-d1dde591ec92"}]': finished 2026-02-13T02:21:08.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:07 trial027 ceph-mon[24567]: osd.4 [v2:10.20.193.28:6808/711804032,v1:10.20.193.28:6809/711804032] boot 2026-02-13T02:21:08.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:07 trial027 ceph-mon[24567]: osdmap e26: 6 total, 5 up, 6 in 2026-02-13T02:21:08.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:07 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-13T02:21:08.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:07 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-13T02:21:08.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:07 trial027 ceph-mon[24567]: from='client.? 10.20.193.107:0/2418694110' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bb16754e-66f2-4951-9f31-d1dde591ec92"} : dispatch 2026-02-13T02:21:08.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:07 trial027 ceph-mon[24567]: from='client.? 10.20.193.107:0/1929671483' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:21:08.411 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:07 trial027 ceph-mon[24567]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:08.411 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:07 trial027 ceph-mon[24567]: Cluster is now healthy 2026-02-13T02:21:09.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:09 trial027 ceph-mon[24567]: purged_snaps scrub starts 2026-02-13T02:21:09.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:09 trial027 ceph-mon[24567]: purged_snaps scrub ok 2026-02-13T02:21:09.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:09 trial027 ceph-mon[24567]: osdmap e27: 6 total, 5 up, 6 in 2026-02-13T02:21:09.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:09 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-13T02:21:09.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:09 trial027 ceph-mon[24567]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 3.4 TiB / 3.4 TiB avail; 199 KiB/s, 0 objects/s recovering 2026-02-13T02:21:09.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:09 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:09.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:09 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:09.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:09 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:09.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:09 trial028 ceph-mon[26701]: purged_snaps scrub starts 2026-02-13T02:21:09.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:09 trial028 ceph-mon[26701]: purged_snaps scrub ok 2026-02-13T02:21:09.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:09 trial028 ceph-mon[26701]: osdmap e27: 6 total, 5 up, 6 in 2026-02-13T02:21:09.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:09 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-13T02:21:09.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:09 trial028 ceph-mon[26701]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 3.4 TiB / 3.4 TiB avail; 199 KiB/s, 0 objects/s recovering 2026-02-13T02:21:09.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:09 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:09.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:09 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:09.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:09 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:09.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:09 trial107 ceph-mon[27422]: purged_snaps scrub starts 2026-02-13T02:21:09.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:09 trial107 ceph-mon[27422]: purged_snaps scrub ok 2026-02-13T02:21:09.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:09 trial107 ceph-mon[27422]: osdmap e27: 6 total, 5 up, 6 in 2026-02-13T02:21:09.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:09 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-13T02:21:09.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:09 trial107 ceph-mon[27422]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 3.4 TiB / 3.4 TiB avail; 199 KiB/s, 0 objects/s recovering 2026-02-13T02:21:09.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:09 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:09.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:09 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:09.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:09 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:10.208 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:10 trial107 ceph-mon[27422]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 3.4 TiB / 3.4 TiB avail; 193 KiB/s, 0 objects/s recovering 2026-02-13T02:21:10.208 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:10 trial107 ceph-mon[27422]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:10.208 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:10 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2026-02-13T02:21:10.208 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:10 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:10.208 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:10 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:10.208 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:10 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:10.208 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:10 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:10.209 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:10 trial107 ceph-mon[27422]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-13T02:21:10.209 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:10 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:10.209 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:10 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:10.209 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:10 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:10.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:10 trial027 ceph-mon[24567]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 3.4 TiB / 3.4 TiB avail; 193 KiB/s, 0 objects/s recovering 2026-02-13T02:21:10.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:10 trial027 ceph-mon[24567]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:10.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:10 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2026-02-13T02:21:10.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:10 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:10.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:10 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:10.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:10 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:10.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:10 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:10.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:10 trial027 ceph-mon[24567]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-13T02:21:10.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:10 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:10.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:10 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:10.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:10 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:10.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:10 trial028 ceph-mon[26701]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 3.4 TiB / 3.4 TiB avail; 193 KiB/s, 0 objects/s recovering 2026-02-13T02:21:10.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:10 trial028 ceph-mon[26701]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:10.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:10 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2026-02-13T02:21:10.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:10 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:10.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:10 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:10.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:10 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:10.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:10 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:10.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:10 trial028 ceph-mon[26701]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-13T02:21:10.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:10 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:10.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:10 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:10.495 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:10 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:11.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:11 trial107 ceph-mon[27422]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:11.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:11 trial107 ceph-mon[27422]: Cluster is now healthy 2026-02-13T02:21:11.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:11 trial027 ceph-mon[24567]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:11.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:11 trial027 ceph-mon[24567]: Cluster is now healthy 2026-02-13T02:21:11.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:11 trial028 ceph-mon[26701]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:11.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:11 trial028 ceph-mon[26701]: Cluster is now healthy 2026-02-13T02:21:12.317 INFO:teuthology.orchestra.run.trial107.stdout:Created osd(s) 5 on host 'trial107' 2026-02-13T02:21:12.317 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:12.319+0000 7f309e7fc6c0 1 -- 10.20.193.107:0/4023313429 <== mgr.14116 v2:10.20.193.27:6800/1837452956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7f30b0061dd0 con 0x7f308c0b5600 2026-02-13T02:21:12.320 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:12.321+0000 7f30b73f46c0 1 -- 10.20.193.107:0/4023313429 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f308c0b5600 msgr2=0x7f308c0d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:12.320 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:12.321+0000 7f30b73f46c0 1 --2- 10.20.193.107:0/4023313429 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f308c0b5600 0x7f308c0d5ab0 secure :-1 s=READY pgs=101 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f30a0004750 tx=0x7f30a00095b0 comp rx=0 tx=0).stop 2026-02-13T02:21:12.320 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:12.322+0000 7f30b73f46c0 1 -- 10.20.193.107:0/4023313429 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f30b010e780 msgr2=0x7f30b00f8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:12.320 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:12.322+0000 7f30b73f46c0 1 --2- 10.20.193.107:0/4023313429 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f30b010e780 0x7f30b00f8ea0 secure :-1 s=READY pgs=18 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f30ac00c6c0 tx=0x7f30ac00cb90 comp rx=0 tx=0).stop 2026-02-13T02:21:12.320 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:12.322+0000 7f30b73f46c0 1 -- 10.20.193.107:0/4023313429 shutdown_connections 2026-02-13T02:21:12.320 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:12.322+0000 7f30b73f46c0 1 --2- 10.20.193.107:0/4023313429 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f308c0b5600 0x7f308c0d5ab0 unknown :-1 s=CLOSED pgs=101 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:12.320 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:12.322+0000 7f30b73f46c0 1 --2- 10.20.193.107:0/4023313429 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f30b010e780 0x7f30b00f8ea0 unknown :-1 s=CLOSED pgs=18 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:12.320 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:12.322+0000 7f30b73f46c0 1 --2- 10.20.193.107:0/4023313429 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f30b010ddd0 0x7f30b016f110 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:12.320 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:12.322+0000 7f30b73f46c0 1 --2- 10.20.193.107:0/4023313429 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f30b010cc50 0x7f30b00f8640 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:12.321 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:12.322+0000 7f30b73f46c0 1 -- 10.20.193.107:0/4023313429 >> 10.20.193.107:0/4023313429 conn(0x7f30b00ea370 msgr2=0x7f30b00616c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:21:12.321 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:12.322+0000 7f30b73f46c0 1 -- 10.20.193.107:0/4023313429 shutdown_connections 2026-02-13T02:21:12.321 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:12.322+0000 7f30b73f46c0 1 -- 10.20.193.107:0/4023313429 wait complete. 2026-02-13T02:21:12.360 DEBUG:teuthology.orchestra.run.trial107:osd.5> sudo journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.5.service 2026-02-13T02:21:12.362 INFO:tasks.cephadm:Deploying osd.6 on trial027 with /dev/vg_nvme/lv_2... 2026-02-13T02:21:12.363 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_2 2026-02-13T02:21:12.480 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:21:12.535 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:12 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:12.536 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:12 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:12.536 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:12 trial027 ceph-mon[24567]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-13T02:21:12.536 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:12 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:12.536 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:12 trial027 ceph-mon[24567]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2026-02-13T02:21:12.536 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:12 trial027 ceph-mon[24567]: from='osd.5 [v2:10.20.193.107:6808/4246100128,v1:10.20.193.107:6809/4246100128]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2026-02-13T02:21:12.536 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:12 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:12.536 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:12 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:12.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:12 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:12.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:12 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:12.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:12 trial028 ceph-mon[26701]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-13T02:21:12.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:12 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:12.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:12 trial028 ceph-mon[26701]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2026-02-13T02:21:12.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:12 trial028 ceph-mon[26701]: from='osd.5 [v2:10.20.193.107:6808/4246100128,v1:10.20.193.107:6809/4246100128]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2026-02-13T02:21:12.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:12 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:12.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:12 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:12.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:12 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:12.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:12 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:12.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:12 trial107 ceph-mon[27422]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-13T02:21:12.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:12 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:12.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:12 trial107 ceph-mon[27422]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2026-02-13T02:21:12.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:12 trial107 ceph-mon[27422]: from='osd.5 [v2:10.20.193.107:6808/4246100128,v1:10.20.193.107:6809/4246100128]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2026-02-13T02:21:12.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:12 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:12.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:12 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:12.927 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:21:12.944 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph orch daemon add osd trial027:vg_nvme/lv_2 --skip-validation 2026-02-13T02:21:13.062 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:21:13.191 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.186+0000 7fe9011336c0 1 -- 10.20.193.27:0/717849918 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe8fc110050 msgr2=0x7fe8fc1104c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:13.191 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.186+0000 7fe9011336c0 1 --2- 10.20.193.27:0/717849918 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe8fc110050 0x7fe8fc1104c0 secure :-1 s=READY pgs=162 gs=1 cs=0 l=1 c_cookie=8599292103498937883 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe8e4009970 tx=0x7fe8e402ed20 comp rx=0 tx=0).stop 2026-02-13T02:21:13.191 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.186+0000 7fe9011336c0 1 -- 10.20.193.27:0/717849918 shutdown_connections 2026-02-13T02:21:13.191 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.186+0000 7fe9011336c0 1 --2- 10.20.193.27:0/717849918 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe8fc110a00 0x7fe8fc171390 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:13.191 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.186+0000 7fe9011336c0 1 --2- 10.20.193.27:0/717849918 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe8fc110050 0x7fe8fc1104c0 unknown :-1 s=CLOSED pgs=162 gs=1 cs=0 l=1 c_cookie=8599292103498937883 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:13.191 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.186+0000 7fe9011336c0 1 --2- 10.20.193.27:0/717849918 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe8fc10eed0 0x7fe8fc10f2c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:13.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.186+0000 7fe9011336c0 1 -- 10.20.193.27:0/717849918 >> 10.20.193.27:0/717849918 conn(0x7fe8fc0ec5f0 msgr2=0x7fe8fc10ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:21:13.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.186+0000 7fe9011336c0 1 -- 10.20.193.27:0/717849918 shutdown_connections 2026-02-13T02:21:13.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.186+0000 7fe9011336c0 1 -- 10.20.193.27:0/717849918 wait complete. 2026-02-13T02:21:13.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.186+0000 7fe9011336c0 1 Processor -- start 2026-02-13T02:21:13.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.186+0000 7fe9011336c0 1 -- start start 2026-02-13T02:21:13.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.186+0000 7fe9011336c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe8fc10eed0 0x7fe8fc0f8600 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:13.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.186+0000 7fe9011336c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe8fc110050 0x7fe8fc171390 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:13.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.186+0000 7fe9011336c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe8fc110a00 0x7fe8fc0f8e60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:13.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.186+0000 7fe9011336c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc0f9a90 con 0x7fe8fc110050 2026-02-13T02:21:13.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.186+0000 7fe9011336c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc0f9c00 con 0x7fe8fc110a00 2026-02-13T02:21:13.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.186+0000 7fe9011336c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc0f9dc0 con 0x7fe8fc10eed0 2026-02-13T02:21:13.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.186+0000 7fe8fa5756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe8fc110050 0x7fe8fc171390 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:13.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.187+0000 7fe8fa5756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe8fc110050 0x7fe8fc171390 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:39886/0 (socket says 10.20.193.27:39886) 2026-02-13T02:21:13.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.187+0000 7fe8fa5756c0 1 -- 10.20.193.27:0/2146611074 learned_addr learned my addr 10.20.193.27:0/2146611074 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:21:13.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.187+0000 7fe8fad766c0 1 --2- 10.20.193.27:0/2146611074 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe8fc10eed0 0x7fe8fc0f8600 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:13.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.187+0000 7fe8fb5776c0 1 --2- 10.20.193.27:0/2146611074 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe8fc110a00 0x7fe8fc0f8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:13.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.187+0000 7fe8fa5756c0 1 -- 10.20.193.27:0/2146611074 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe8fc10eed0 msgr2=0x7fe8fc0f8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:13.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.187+0000 7fe8fa5756c0 1 --2- 10.20.193.27:0/2146611074 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe8fc10eed0 0x7fe8fc0f8600 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:13.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.187+0000 7fe8fa5756c0 1 -- 10.20.193.27:0/2146611074 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe8fc110a00 msgr2=0x7fe8fc0f8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:13.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.187+0000 7fe8fa5756c0 1 --2- 10.20.193.27:0/2146611074 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe8fc110a00 0x7fe8fc0f8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:13.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.187+0000 7fe8fa5756c0 1 -- 10.20.193.27:0/2146611074 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e4009590 con 0x7fe8fc110050 2026-02-13T02:21:13.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.187+0000 7fe8fad766c0 1 --2- 10.20.193.27:0/2146611074 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe8fc10eed0 0x7fe8fc0f8600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:21:13.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.187+0000 7fe8fa5756c0 1 --2- 10.20.193.27:0/2146611074 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe8fc110050 0x7fe8fc171390 secure :-1 s=READY pgs=163 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe8e40092e0 tx=0x7fe8e4004670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:21:13.194 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.187+0000 7fe8dbfff6c0 1 -- 10.20.193.27:0/2146611074 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe8e4002ae0 con 0x7fe8fc110050 2026-02-13T02:21:13.194 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.187+0000 7fe8dbfff6c0 1 -- 10.20.193.27:0/2146611074 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fe8e4002c40 con 0x7fe8fc110050 2026-02-13T02:21:13.194 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.187+0000 7fe9011336c0 1 -- 10.20.193.27:0/2146611074 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8fc0fa040 con 0x7fe8fc110050 2026-02-13T02:21:13.194 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.187+0000 7fe8dbfff6c0 1 -- 10.20.193.27:0/2146611074 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe8e40372f0 con 0x7fe8fc110050 2026-02-13T02:21:13.194 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.187+0000 7fe9011336c0 1 -- 10.20.193.27:0/2146611074 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8fc0fa4c0 con 0x7fe8fc110050 2026-02-13T02:21:13.194 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.187+0000 7fe9011336c0 1 -- 10.20.193.27:0/2146611074 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8fc0fa6e0 con 0x7fe8fc110050 2026-02-13T02:21:13.195 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.189+0000 7fe8dbfff6c0 1 -- 10.20.193.27:0/2146611074 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7fe8e4007930 con 0x7fe8fc110050 2026-02-13T02:21:13.195 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.190+0000 7fe8dbfff6c0 1 --2- 10.20.193.27:0/2146611074 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe8d00b5600 0x7fe8d00d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:13.195 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.190+0000 7fe8dbfff6c0 1 -- 10.20.193.27:0/2146611074 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4014+0+0 (secure 0 0 0) 0x7fe8e4106d80 con 0x7fe8fc110050 2026-02-13T02:21:13.195 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.190+0000 7fe8fad766c0 1 --2- 10.20.193.27:0/2146611074 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe8d00b5600 0x7fe8d00d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:13.195 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.190+0000 7fe8fad766c0 1 --2- 10.20.193.27:0/2146611074 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe8d00b5600 0x7fe8d00d5ab0 secure :-1 s=READY pgs=106 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe8f0004750 tx=0x7fe8f0009260 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:21:13.196 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.190+0000 7fe8dbfff6c0 1 -- 10.20.193.27:0/2146611074 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fe8e40ca7b0 con 0x7fe8fc110050 2026-02-13T02:21:13.289 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:13.286+0000 7fe9011336c0 1 -- 10.20.193.27:0/2146611074 --> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "trial027:vg_nvme/lv_2", "skip_validation": true, "target": ["mon-mgr", ""]}) -- 0x7fe8fc174bd0 con 0x7fe8d00b5600 2026-02-13T02:21:13.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:13 trial027 ceph-mon[24567]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-13T02:21:13.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:13 trial027 ceph-mon[24567]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:13.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:13 trial027 ceph-mon[24567]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2026-02-13T02:21:13.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:13 trial027 ceph-mon[24567]: osdmap e28: 6 total, 5 up, 6 in 2026-02-13T02:21:13.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:13 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-13T02:21:13.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:13 trial027 ceph-mon[24567]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.6836, "args": ["host=trial107", "root=default"]} : dispatch 2026-02-13T02:21:13.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:13 trial027 ceph-mon[24567]: from='osd.5 [v2:10.20.193.107:6808/4246100128,v1:10.20.193.107:6809/4246100128]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.6836, "args": ["host=trial107", "root=default"]} : dispatch 2026-02-13T02:21:13.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:13 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:13.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:13 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:13.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:13 trial028 ceph-mon[26701]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-13T02:21:13.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:13 trial028 ceph-mon[26701]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:13.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:13 trial028 ceph-mon[26701]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2026-02-13T02:21:13.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:13 trial028 ceph-mon[26701]: osdmap e28: 6 total, 5 up, 6 in 2026-02-13T02:21:13.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:13 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-13T02:21:13.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:13 trial028 ceph-mon[26701]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.6836, "args": ["host=trial107", "root=default"]} : dispatch 2026-02-13T02:21:13.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:13 trial028 ceph-mon[26701]: from='osd.5 [v2:10.20.193.107:6808/4246100128,v1:10.20.193.107:6809/4246100128]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.6836, "args": ["host=trial107", "root=default"]} : dispatch 2026-02-13T02:21:13.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:13 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:13.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:13 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:13.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:13 trial107 ceph-mon[27422]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-13T02:21:13.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:13 trial107 ceph-mon[27422]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:13.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:13 trial107 ceph-mon[27422]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2026-02-13T02:21:13.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:13 trial107 ceph-mon[27422]: osdmap e28: 6 total, 5 up, 6 in 2026-02-13T02:21:13.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:13 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-13T02:21:13.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:13 trial107 ceph-mon[27422]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.6836, "args": ["host=trial107", "root=default"]} : dispatch 2026-02-13T02:21:13.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:13 trial107 ceph-mon[27422]: from='osd.5 [v2:10.20.193.107:6808/4246100128,v1:10.20.193.107:6809/4246100128]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.6836, "args": ["host=trial107", "root=default"]} : dispatch 2026-02-13T02:21:13.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:13 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:13.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:13 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:13.751 INFO:journalctl@ceph.osd.5.trial107.stdout:Feb 13 02:21:13 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-5[38600]: 2026-02-13T02:21:13.297+0000 7ff20fc8c6c0 -1 osd.5 0 waiting for initial osdmap 2026-02-13T02:21:13.752 INFO:journalctl@ceph.osd.5.trial107.stdout:Feb 13 02:21:13 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-5[38600]: 2026-02-13T02:21:13.301+0000 7ff20bb506c0 -1 osd.5 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-13T02:21:14.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:14 trial027 ceph-mon[24567]: from='client.14502 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial027:vg_nvme/lv_2", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:21:14.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:14 trial027 ceph-mon[24567]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.6836, "args": ["host=trial107", "root=default"]}]': finished 2026-02-13T02:21:14.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:14 trial027 ceph-mon[24567]: osdmap e29: 6 total, 5 up, 6 in 2026-02-13T02:21:14.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-13T02:21:14.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:14.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:14.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-13T02:21:14.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:14.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:14.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:14.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:14.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:14.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:14.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:14.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:14.661 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:14.661 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:14.661 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:14.661 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:14 trial027 ceph-mon[24567]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-13T02:21:14.661 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:14.661 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:14.661 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:14.661 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:14 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3843216955' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "318afff7-b7c9-4b19-a7ba-06e7ab122ab6"} : dispatch 2026-02-13T02:21:14.661 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:14 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3843216955' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "318afff7-b7c9-4b19-a7ba-06e7ab122ab6"}]': finished 2026-02-13T02:21:14.661 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:14 trial027 ceph-mon[24567]: osd.5 [v2:10.20.193.107:6808/4246100128,v1:10.20.193.107:6809/4246100128] boot 2026-02-13T02:21:14.662 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:14 trial027 ceph-mon[24567]: osdmap e30: 7 total, 6 up, 7 in 2026-02-13T02:21:14.662 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-13T02:21:14.662 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-13T02:21:14.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:14 trial028 ceph-mon[26701]: from='client.14502 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial027:vg_nvme/lv_2", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:21:14.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:14 trial028 ceph-mon[26701]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.6836, "args": ["host=trial107", "root=default"]}]': finished 2026-02-13T02:21:14.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:14 trial028 ceph-mon[26701]: osdmap e29: 6 total, 5 up, 6 in 2026-02-13T02:21:14.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-13T02:21:14.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:14.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:14.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-13T02:21:14.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:14.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:14.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:14.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:14.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:14.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:14.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:14.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:14.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:14.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:14.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:14.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:14 trial028 ceph-mon[26701]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-13T02:21:14.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:14.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:14.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:14.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:14 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3843216955' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "318afff7-b7c9-4b19-a7ba-06e7ab122ab6"} : dispatch 2026-02-13T02:21:14.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:14 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3843216955' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "318afff7-b7c9-4b19-a7ba-06e7ab122ab6"}]': finished 2026-02-13T02:21:14.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:14 trial028 ceph-mon[26701]: osd.5 [v2:10.20.193.107:6808/4246100128,v1:10.20.193.107:6809/4246100128] boot 2026-02-13T02:21:14.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:14 trial028 ceph-mon[26701]: osdmap e30: 7 total, 6 up, 7 in 2026-02-13T02:21:14.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-13T02:21:14.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-13T02:21:14.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:14 trial107 ceph-mon[27422]: from='client.14502 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial027:vg_nvme/lv_2", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:21:14.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:14 trial107 ceph-mon[27422]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.6836, "args": ["host=trial107", "root=default"]}]': finished 2026-02-13T02:21:14.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:14 trial107 ceph-mon[27422]: osdmap e29: 6 total, 5 up, 6 in 2026-02-13T02:21:14.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-13T02:21:14.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:14.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:14.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-13T02:21:14.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:14.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:14.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:14.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:14.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:14.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:14.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:14.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:14.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:14.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:14.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:14.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:14 trial107 ceph-mon[27422]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-13T02:21:14.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:14.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:14.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:14.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:14 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3843216955' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "318afff7-b7c9-4b19-a7ba-06e7ab122ab6"} : dispatch 2026-02-13T02:21:14.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:14 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3843216955' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "318afff7-b7c9-4b19-a7ba-06e7ab122ab6"}]': finished 2026-02-13T02:21:14.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:14 trial107 ceph-mon[27422]: osd.5 [v2:10.20.193.107:6808/4246100128,v1:10.20.193.107:6809/4246100128] boot 2026-02-13T02:21:14.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:14 trial107 ceph-mon[27422]: osdmap e30: 7 total, 6 up, 7 in 2026-02-13T02:21:14.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-13T02:21:14.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-13T02:21:15.562 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:15 trial027 ceph-mon[24567]: purged_snaps scrub starts 2026-02-13T02:21:15.562 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:15 trial027 ceph-mon[24567]: purged_snaps scrub ok 2026-02-13T02:21:15.562 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:15 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2634111243' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:21:15.562 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:15 trial027 ceph-mon[24567]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:15.562 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:15 trial027 ceph-mon[24567]: Cluster is now healthy 2026-02-13T02:21:15.562 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:15 trial027 ceph-mon[24567]: osdmap e31: 7 total, 6 up, 7 in 2026-02-13T02:21:15.562 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:15 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-13T02:21:15.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:15 trial028 ceph-mon[26701]: purged_snaps scrub starts 2026-02-13T02:21:15.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:15 trial028 ceph-mon[26701]: purged_snaps scrub ok 2026-02-13T02:21:15.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:15 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2634111243' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:21:15.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:15 trial028 ceph-mon[26701]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:15.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:15 trial028 ceph-mon[26701]: Cluster is now healthy 2026-02-13T02:21:15.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:15 trial028 ceph-mon[26701]: osdmap e31: 7 total, 6 up, 7 in 2026-02-13T02:21:15.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:15 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-13T02:21:15.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:15 trial107 ceph-mon[27422]: purged_snaps scrub starts 2026-02-13T02:21:15.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:15 trial107 ceph-mon[27422]: purged_snaps scrub ok 2026-02-13T02:21:15.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:15 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2634111243' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:21:15.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:15 trial107 ceph-mon[27422]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:15.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:15 trial107 ceph-mon[27422]: Cluster is now healthy 2026-02-13T02:21:15.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:15 trial107 ceph-mon[27422]: osdmap e31: 7 total, 6 up, 7 in 2026-02-13T02:21:15.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:15 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-13T02:21:16.385 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:16 trial027 ceph-mon[24567]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-13T02:21:16.385 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:16 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:16.385 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:16 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:16.385 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:16 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:16.385 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:16 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:16.385 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:16 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:16.385 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:16 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:16.385 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:16 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:16.386 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:16 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:16.386 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:16 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:16.386 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:16 trial027 ceph-mon[24567]: osdmap e32: 7 total, 6 up, 7 in 2026-02-13T02:21:16.386 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:16 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-13T02:21:16.386 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:16 trial027 ceph-mon[24567]: osdmap e33: 7 total, 6 up, 7 in 2026-02-13T02:21:16.386 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:16 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-13T02:21:16.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:16 trial028 ceph-mon[26701]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-13T02:21:16.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:16 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:16.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:16 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:16.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:16 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:16.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:16 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:16.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:16 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:16.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:16 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:16.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:16 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:16.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:16 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:16.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:16 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:16.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:16 trial028 ceph-mon[26701]: osdmap e32: 7 total, 6 up, 7 in 2026-02-13T02:21:16.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:16 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-13T02:21:16.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:16 trial028 ceph-mon[26701]: osdmap e33: 7 total, 6 up, 7 in 2026-02-13T02:21:16.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:16 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-13T02:21:16.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:16 trial107 ceph-mon[27422]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-13T02:21:16.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:16 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:16.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:16 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:16.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:16 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:16.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:16 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:16.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:16 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:16.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:16 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:16.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:16 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:16.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:16 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:16.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:16 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:16.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:16 trial107 ceph-mon[27422]: osdmap e32: 7 total, 6 up, 7 in 2026-02-13T02:21:16.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:16 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-13T02:21:16.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:16 trial107 ceph-mon[27422]: osdmap e33: 7 total, 6 up, 7 in 2026-02-13T02:21:16.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:16 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-13T02:21:17.473 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:17 trial027 ceph-mon[24567]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-13T02:21:17.474 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:17 trial027 ceph-mon[24567]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-13T02:21:17.474 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:17 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2026-02-13T02:21:17.474 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:17 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:17.474 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:17 trial027 ceph-mon[24567]: osdmap e34: 7 total, 6 up, 7 in 2026-02-13T02:21:17.474 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:17 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-13T02:21:17.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:17 trial028 ceph-mon[26701]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-13T02:21:17.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:17 trial028 ceph-mon[26701]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-13T02:21:17.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:17 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2026-02-13T02:21:17.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:17 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:17.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:17 trial028 ceph-mon[26701]: osdmap e34: 7 total, 6 up, 7 in 2026-02-13T02:21:17.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:17 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-13T02:21:17.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:17 trial107 ceph-mon[27422]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-13T02:21:17.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:17 trial107 ceph-mon[27422]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-13T02:21:17.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:17 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2026-02-13T02:21:17.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:17 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:17.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:17 trial107 ceph-mon[27422]: osdmap e34: 7 total, 6 up, 7 in 2026-02-13T02:21:17.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:17 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-13T02:21:18.651 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:18 trial107 ceph-mon[27422]: pgmap v89: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 4.1 TiB / 4.1 TiB avail; 358 KiB/s, 1 objects/s recovering 2026-02-13T02:21:18.651 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:18 trial107 ceph-mon[27422]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:18.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:18 trial027 ceph-mon[24567]: pgmap v89: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 4.1 TiB / 4.1 TiB avail; 358 KiB/s, 1 objects/s recovering 2026-02-13T02:21:18.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:18 trial027 ceph-mon[24567]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:18.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:18 trial028 ceph-mon[26701]: pgmap v89: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 4.1 TiB / 4.1 TiB avail; 358 KiB/s, 1 objects/s recovering 2026-02-13T02:21:18.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:18 trial028 ceph-mon[26701]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:19.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:19 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:19.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:19 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:19.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:19 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:19.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:19 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:19.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:19 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:19.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:19 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:19.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:19 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:19.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:19 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:19.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:19 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:19.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:19 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:19.410 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:19 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:19.604 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:19.603+0000 7fe8dbfff6c0 1 -- 10.20.193.27:0/2146611074 <== mgr.14116 v2:10.20.193.27:6800/1837452956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7fe8fc174bd0 con 0x7fe8d00b5600 2026-02-13T02:21:19.605 INFO:teuthology.orchestra.run.trial027.stdout:Created osd(s) 6 on host 'trial027' 2026-02-13T02:21:19.607 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:19.605+0000 7fe9011336c0 1 -- 10.20.193.27:0/2146611074 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe8d00b5600 msgr2=0x7fe8d00d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:19.607 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:19.605+0000 7fe9011336c0 1 --2- 10.20.193.27:0/2146611074 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe8d00b5600 0x7fe8d00d5ab0 secure :-1 s=READY pgs=106 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe8f0004750 tx=0x7fe8f0009260 comp rx=0 tx=0).stop 2026-02-13T02:21:19.607 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:19.605+0000 7fe9011336c0 1 -- 10.20.193.27:0/2146611074 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe8fc110050 msgr2=0x7fe8fc171390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:19.607 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:19.605+0000 7fe9011336c0 1 --2- 10.20.193.27:0/2146611074 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe8fc110050 0x7fe8fc171390 secure :-1 s=READY pgs=163 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe8e40092e0 tx=0x7fe8e4004670 comp rx=0 tx=0).stop 2026-02-13T02:21:19.607 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:19.605+0000 7fe9011336c0 1 -- 10.20.193.27:0/2146611074 shutdown_connections 2026-02-13T02:21:19.607 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:19.605+0000 7fe9011336c0 1 --2- 10.20.193.27:0/2146611074 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe8d00b5600 0x7fe8d00d5ab0 unknown :-1 s=CLOSED pgs=106 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:19.608 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:19.605+0000 7fe9011336c0 1 --2- 10.20.193.27:0/2146611074 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe8fc110a00 0x7fe8fc0f8e60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:19.608 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:19.605+0000 7fe9011336c0 1 --2- 10.20.193.27:0/2146611074 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe8fc110050 0x7fe8fc171390 unknown :-1 s=CLOSED pgs=163 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:19.608 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:19.605+0000 7fe9011336c0 1 --2- 10.20.193.27:0/2146611074 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe8fc10eed0 0x7fe8fc0f8600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:19.608 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:19.605+0000 7fe9011336c0 1 -- 10.20.193.27:0/2146611074 >> 10.20.193.27:0/2146611074 conn(0x7fe8fc0ec5f0 msgr2=0x7fe8fc0eebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:21:19.608 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:19.605+0000 7fe9011336c0 1 -- 10.20.193.27:0/2146611074 shutdown_connections 2026-02-13T02:21:19.608 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:19.605+0000 7fe9011336c0 1 -- 10.20.193.27:0/2146611074 wait complete. 2026-02-13T02:21:19.647 DEBUG:teuthology.orchestra.run.trial027:osd.6> sudo journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.6.service 2026-02-13T02:21:19.650 INFO:tasks.cephadm:Deploying osd.7 on trial028 with /dev/vg_nvme/lv_2... 2026-02-13T02:21:19.650 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_2 2026-02-13T02:21:19.674 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:19 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:19.674 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:19 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:19.674 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:19 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:19.674 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:19 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:19.674 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:19 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:19.674 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:19 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:19.674 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:19 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:19.674 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:19 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:19.675 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:19 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:19.675 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:19 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:19.675 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:19 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:19.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:19 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:19.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:19 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:19.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:19 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:19.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:19 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:19.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:19 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:19.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:19 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:19.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:19 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:19.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:19 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:19.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:19 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:19.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:19 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:19.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:19 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:19.771 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.b/config 2026-02-13T02:21:20.212 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:21:20.230 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph orch daemon add osd trial028:vg_nvme/lv_2 --skip-validation 2026-02-13T02:21:20.388 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.b/config 2026-02-13T02:21:20.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:20 trial028 ceph-mon[26701]: pgmap v91: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 4.1 TiB / 4.1 TiB avail; 215 KiB/s, 0 objects/s recovering 2026-02-13T02:21:20.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:20 trial028 ceph-mon[26701]: pgmap v92: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 4.1 TiB / 4.1 TiB avail; 185 KiB/s, 0 objects/s recovering 2026-02-13T02:21:20.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:20 trial028 ceph-mon[26701]: from='osd.6 [v2:10.20.193.27:6818/3514139178,v1:10.20.193.27:6819/3514139178]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2026-02-13T02:21:20.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:20 trial028 ceph-mon[26701]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:20.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:20 trial028 ceph-mon[26701]: Cluster is now healthy 2026-02-13T02:21:20.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:20 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:20.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:20 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:20.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:20 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:20.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.506+0000 7f5731b816c0 1 -- 10.20.193.28:0/4165900447 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f572c1109f0 msgr2=0x7f572c171380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:20.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.506+0000 7f5731b816c0 1 --2- 10.20.193.28:0/4165900447 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f572c1109f0 0x7f572c171380 secure :-1 s=READY pgs=24 gs=1 cs=0 l=1 c_cookie=17238147663680943414 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f572000aa40 tx=0x7f572002fde0 comp rx=0 tx=0).stop 2026-02-13T02:21:20.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.506+0000 7f5731b816c0 1 -- 10.20.193.28:0/4165900447 shutdown_connections 2026-02-13T02:21:20.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.506+0000 7f5731b816c0 1 --2- 10.20.193.28:0/4165900447 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f572c1109f0 0x7f572c171380 unknown :-1 s=CLOSED pgs=24 gs=1 cs=0 l=1 c_cookie=17238147663680943414 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:20.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.506+0000 7f5731b816c0 1 --2- 10.20.193.28:0/4165900447 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f572c110040 0x7f572c1104b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:20.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.506+0000 7f5731b816c0 1 --2- 10.20.193.28:0/4165900447 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f572c10eec0 0x7f572c10f2b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:20.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.506+0000 7f5731b816c0 1 -- 10.20.193.28:0/4165900447 >> 10.20.193.28:0/4165900447 conn(0x7f572c0ec5c0 msgr2=0x7f572c10c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:21:20.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.506+0000 7f5731b816c0 1 -- 10.20.193.28:0/4165900447 shutdown_connections 2026-02-13T02:21:20.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.506+0000 7f5731b816c0 1 -- 10.20.193.28:0/4165900447 wait complete. 2026-02-13T02:21:20.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.506+0000 7f5731b816c0 1 Processor -- start 2026-02-13T02:21:20.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.506+0000 7f5731b816c0 1 -- start start 2026-02-13T02:21:20.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.506+0000 7f5731b816c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f572c10eec0 0x7f572c0f85e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:20.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.506+0000 7f5731b816c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f572c110040 0x7f572c171380 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:20.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.506+0000 7f5731b816c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f572c1109f0 0x7f572c0f8e40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:20.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.506+0000 7f5731b816c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f572c0f9a70 con 0x7f572c10eec0 2026-02-13T02:21:20.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.506+0000 7f5731b816c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f572c0f9be0 con 0x7f572c110040 2026-02-13T02:21:20.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.506+0000 7f5731b816c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f572c0f9da0 con 0x7f572c1109f0 2026-02-13T02:21:20.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.506+0000 7f572affd6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f572c110040 0x7f572c171380 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:20.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.506+0000 7f572affd6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f572c110040 0x7f572c171380 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58528/0 (socket says 10.20.193.28:58528) 2026-02-13T02:21:20.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.506+0000 7f572affd6c0 1 -- 10.20.193.28:0/426556166 learned_addr learned my addr 10.20.193.28:0/426556166 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-13T02:21:20.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.507+0000 7f572b7fe6c0 1 --2- 10.20.193.28:0/426556166 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f572c10eec0 0x7f572c0f85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:20.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.507+0000 7f572bfff6c0 1 --2- 10.20.193.28:0/426556166 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f572c1109f0 0x7f572c0f8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:20.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.507+0000 7f572affd6c0 1 -- 10.20.193.28:0/426556166 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f572c1109f0 msgr2=0x7f572c0f8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:20.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.507+0000 7f572affd6c0 1 --2- 10.20.193.28:0/426556166 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f572c1109f0 0x7f572c0f8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:20.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.507+0000 7f572affd6c0 1 -- 10.20.193.28:0/426556166 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f572c10eec0 msgr2=0x7f572c0f85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:20.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.507+0000 7f572affd6c0 1 --2- 10.20.193.28:0/426556166 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f572c10eec0 0x7f572c0f85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:20.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.507+0000 7f572affd6c0 1 -- 10.20.193.28:0/426556166 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f572000a660 con 0x7f572c110040 2026-02-13T02:21:20.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.507+0000 7f572affd6c0 1 --2- 10.20.193.28:0/426556166 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f572c110040 0x7f572c171380 secure :-1 s=READY pgs=25 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f571c00b6c0 tx=0x7f571c00bb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:21:20.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.507+0000 7f5728ff96c0 1 -- 10.20.193.28:0/426556166 <== mon.2 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f571c0044a0 con 0x7f572c110040 2026-02-13T02:21:20.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.507+0000 7f5731b816c0 1 -- 10.20.193.28:0/426556166 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f572c0fa080 con 0x7f572c110040 2026-02-13T02:21:20.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.507+0000 7f5728ff96c0 1 -- 10.20.193.28:0/426556166 <== mon.2 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f571c00be50 con 0x7f572c110040 2026-02-13T02:21:20.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.507+0000 7f5728ff96c0 1 -- 10.20.193.28:0/426556166 <== mon.2 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f571c00cd00 con 0x7f572c110040 2026-02-13T02:21:20.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.507+0000 7f5731b816c0 1 -- 10.20.193.28:0/426556166 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f572c0fa560 con 0x7f572c110040 2026-02-13T02:21:20.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.507+0000 7f5731b816c0 1 -- 10.20.193.28:0/426556166 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f572c1752c0 con 0x7f572c110040 2026-02-13T02:21:20.511 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.509+0000 7f5728ff96c0 1 -- 10.20.193.28:0/426556166 <== mon.2 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7f571c0026b0 con 0x7f572c110040 2026-02-13T02:21:20.511 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.509+0000 7f5728ff96c0 1 --2- 10.20.193.28:0/426556166 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f57000b5600 0x7f57000d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:20.511 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.510+0000 7f5728ff96c0 1 -- 10.20.193.28:0/426556166 <== mon.2 v2:10.20.193.28:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4446+0+0 (secure 0 0 0) 0x7f571c0e2b00 con 0x7f572c110040 2026-02-13T02:21:20.511 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.510+0000 7f572b7fe6c0 1 --2- 10.20.193.28:0/426556166 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f57000b5600 0x7f57000d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:20.511 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.510+0000 7f572b7fe6c0 1 --2- 10.20.193.28:0/426556166 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f57000b5600 0x7f57000d5ab0 secure :-1 s=READY pgs=111 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5718005980 tx=0x7f5718005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:21:20.511 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.510+0000 7f5728ff96c0 1 -- 10.20.193.28:0/426556166 <== mon.2 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f571c0a6410 con 0x7f572c110040 2026-02-13T02:21:20.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:20.608+0000 7f5731b816c0 1 -- 10.20.193.28:0/426556166 --> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_2", "skip_validation": true, "target": ["mon-mgr", ""]}) -- 0x7f572c0f14f0 con 0x7f57000b5600 2026-02-13T02:21:20.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:20 trial027 ceph-mon[24567]: pgmap v91: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 4.1 TiB / 4.1 TiB avail; 215 KiB/s, 0 objects/s recovering 2026-02-13T02:21:20.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:20 trial027 ceph-mon[24567]: pgmap v92: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 4.1 TiB / 4.1 TiB avail; 185 KiB/s, 0 objects/s recovering 2026-02-13T02:21:20.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:20 trial027 ceph-mon[24567]: from='osd.6 [v2:10.20.193.27:6818/3514139178,v1:10.20.193.27:6819/3514139178]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2026-02-13T02:21:20.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:20 trial027 ceph-mon[24567]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:20.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:20 trial027 ceph-mon[24567]: Cluster is now healthy 2026-02-13T02:21:20.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:20 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:20.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:20 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:20.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:20 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:20.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:20 trial107 ceph-mon[27422]: pgmap v91: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 4.1 TiB / 4.1 TiB avail; 215 KiB/s, 0 objects/s recovering 2026-02-13T02:21:20.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:20 trial107 ceph-mon[27422]: pgmap v92: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 4.1 TiB / 4.1 TiB avail; 185 KiB/s, 0 objects/s recovering 2026-02-13T02:21:20.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:20 trial107 ceph-mon[27422]: from='osd.6 [v2:10.20.193.27:6818/3514139178,v1:10.20.193.27:6819/3514139178]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2026-02-13T02:21:20.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:20 trial107 ceph-mon[27422]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:20.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:20 trial107 ceph-mon[27422]: Cluster is now healthy 2026-02-13T02:21:20.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:20 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:20.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:20 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:20.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:20 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:21.510 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:21 trial028 ceph-mon[26701]: pgmap v93: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 4.1 TiB / 4.1 TiB avail; 153 KiB/s, 0 objects/s recovering 2026-02-13T02:21:21.510 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:21 trial028 ceph-mon[26701]: from='osd.6 [v2:10.20.193.27:6818/3514139178,v1:10.20.193.27:6819/3514139178]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2026-02-13T02:21:21.510 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:21 trial028 ceph-mon[26701]: osdmap e35: 7 total, 6 up, 7 in 2026-02-13T02:21:21.510 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:21 trial028 ceph-mon[26701]: from='osd.6 [v2:10.20.193.27:6818/3514139178,v1:10.20.193.27:6819/3514139178]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.6836, "args": ["host=trial027", "root=default"]} : dispatch 2026-02-13T02:21:21.510 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:21 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-13T02:21:21.510 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:21 trial028 ceph-mon[26701]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:21.510 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:21 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:21.510 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:21 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:21.510 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:21 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:21.510 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:21 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:21.510 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:21 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:21.510 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:21 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:21.510 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:21 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:21.510 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:21 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:21.510 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:21 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:21.510 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:21 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:21.510 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:21 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:21.510 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:21 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:21.510 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:21 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:21.510 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:21 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:21.510 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:21 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:21.510 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:21 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:21.510 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:21 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:21.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:21 trial027 ceph-mon[24567]: pgmap v93: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 4.1 TiB / 4.1 TiB avail; 153 KiB/s, 0 objects/s recovering 2026-02-13T02:21:21.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:21 trial027 ceph-mon[24567]: from='osd.6 [v2:10.20.193.27:6818/3514139178,v1:10.20.193.27:6819/3514139178]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2026-02-13T02:21:21.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:21 trial027 ceph-mon[24567]: osdmap e35: 7 total, 6 up, 7 in 2026-02-13T02:21:21.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:21 trial027 ceph-mon[24567]: from='osd.6 [v2:10.20.193.27:6818/3514139178,v1:10.20.193.27:6819/3514139178]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.6836, "args": ["host=trial027", "root=default"]} : dispatch 2026-02-13T02:21:21.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:21 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-13T02:21:21.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:21 trial027 ceph-mon[24567]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:21.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:21 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:21.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:21 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:21.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:21 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:21.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:21 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:21.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:21 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:21.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:21 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:21.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:21 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:21.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:21 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:21.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:21 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:21.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:21 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:21.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:21 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:21.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:21 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:21.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:21 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:21.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:21 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:21.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:21 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:21.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:21 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:21.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:21 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:21.659 INFO:journalctl@ceph.osd.6.trial027.stdout:Feb 13 02:21:21 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-6[49552]: 2026-02-13T02:21:21.313+0000 7f7102d886c0 -1 osd.6 0 waiting for initial osdmap 2026-02-13T02:21:21.659 INFO:journalctl@ceph.osd.6.trial027.stdout:Feb 13 02:21:21 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-6[49552]: 2026-02-13T02:21:21.317+0000 7f70fec4c6c0 -1 osd.6 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-13T02:21:21.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:21 trial107 ceph-mon[27422]: pgmap v93: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 4.1 TiB / 4.1 TiB avail; 153 KiB/s, 0 objects/s recovering 2026-02-13T02:21:21.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:21 trial107 ceph-mon[27422]: from='osd.6 [v2:10.20.193.27:6818/3514139178,v1:10.20.193.27:6819/3514139178]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2026-02-13T02:21:21.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:21 trial107 ceph-mon[27422]: osdmap e35: 7 total, 6 up, 7 in 2026-02-13T02:21:21.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:21 trial107 ceph-mon[27422]: from='osd.6 [v2:10.20.193.27:6818/3514139178,v1:10.20.193.27:6819/3514139178]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.6836, "args": ["host=trial027", "root=default"]} : dispatch 2026-02-13T02:21:21.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:21 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-13T02:21:21.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:21 trial107 ceph-mon[27422]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:21.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:21 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:21.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:21 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:21.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:21 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:21.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:21 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:21.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:21 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:21.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:21 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:21.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:21 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:21.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:21 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:21.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:21 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:21.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:21 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:21.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:21 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:21.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:21 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:21.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:21 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:21.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:21 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:21.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:21 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:21.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:21 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:21.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:21 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:22.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:22 trial027 ceph-mon[24567]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_2", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:21:22.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:22 trial027 ceph-mon[24567]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-13T02:21:22.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:22 trial027 ceph-mon[24567]: from='osd.6 [v2:10.20.193.27:6818/3514139178,v1:10.20.193.27:6819/3514139178]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.6836, "args": ["host=trial027", "root=default"]}]': finished 2026-02-13T02:21:22.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:22 trial027 ceph-mon[24567]: osdmap e36: 7 total, 6 up, 7 in 2026-02-13T02:21:22.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:22 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-13T02:21:22.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:22 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-13T02:21:22.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:22 trial027 ceph-mon[24567]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b545439a-3f32-4b0a-bc6a-17f0d8050c07"} : dispatch 2026-02-13T02:21:22.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:22 trial027 ceph-mon[24567]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b545439a-3f32-4b0a-bc6a-17f0d8050c07"}]': finished 2026-02-13T02:21:22.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:22 trial027 ceph-mon[24567]: osd.6 [v2:10.20.193.27:6818/3514139178,v1:10.20.193.27:6819/3514139178] boot 2026-02-13T02:21:22.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:22 trial027 ceph-mon[24567]: osdmap e37: 8 total, 7 up, 8 in 2026-02-13T02:21:22.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:22 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-13T02:21:22.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:22 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-13T02:21:22.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:22 trial027 ceph-mon[24567]: from='client.? 10.20.193.28:0/2381718069' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b545439a-3f32-4b0a-bc6a-17f0d8050c07"} : dispatch 2026-02-13T02:21:22.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:22 trial027 ceph-mon[24567]: from='client.? 10.20.193.28:0/1805716535' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:21:22.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:22 trial027 ceph-mon[24567]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:22.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:22 trial027 ceph-mon[24567]: Cluster is now healthy 2026-02-13T02:21:22.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:22 trial028 ceph-mon[26701]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_2", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:21:22.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:22 trial028 ceph-mon[26701]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-13T02:21:22.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:22 trial028 ceph-mon[26701]: from='osd.6 [v2:10.20.193.27:6818/3514139178,v1:10.20.193.27:6819/3514139178]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.6836, "args": ["host=trial027", "root=default"]}]': finished 2026-02-13T02:21:22.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:22 trial028 ceph-mon[26701]: osdmap e36: 7 total, 6 up, 7 in 2026-02-13T02:21:22.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:22 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-13T02:21:22.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:22 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-13T02:21:22.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:22 trial028 ceph-mon[26701]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b545439a-3f32-4b0a-bc6a-17f0d8050c07"} : dispatch 2026-02-13T02:21:22.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:22 trial028 ceph-mon[26701]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b545439a-3f32-4b0a-bc6a-17f0d8050c07"}]': finished 2026-02-13T02:21:22.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:22 trial028 ceph-mon[26701]: osd.6 [v2:10.20.193.27:6818/3514139178,v1:10.20.193.27:6819/3514139178] boot 2026-02-13T02:21:22.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:22 trial028 ceph-mon[26701]: osdmap e37: 8 total, 7 up, 8 in 2026-02-13T02:21:22.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:22 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-13T02:21:22.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:22 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-13T02:21:22.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:22 trial028 ceph-mon[26701]: from='client.? 10.20.193.28:0/2381718069' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b545439a-3f32-4b0a-bc6a-17f0d8050c07"} : dispatch 2026-02-13T02:21:22.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:22 trial028 ceph-mon[26701]: from='client.? 10.20.193.28:0/1805716535' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:21:22.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:22 trial028 ceph-mon[26701]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:22.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:22 trial028 ceph-mon[26701]: Cluster is now healthy 2026-02-13T02:21:22.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:22 trial107 ceph-mon[27422]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_2", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:21:22.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:22 trial107 ceph-mon[27422]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-13T02:21:22.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:22 trial107 ceph-mon[27422]: from='osd.6 [v2:10.20.193.27:6818/3514139178,v1:10.20.193.27:6819/3514139178]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.6836, "args": ["host=trial027", "root=default"]}]': finished 2026-02-13T02:21:22.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:22 trial107 ceph-mon[27422]: osdmap e36: 7 total, 6 up, 7 in 2026-02-13T02:21:22.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:22 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-13T02:21:22.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:22 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-13T02:21:22.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:22 trial107 ceph-mon[27422]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b545439a-3f32-4b0a-bc6a-17f0d8050c07"} : dispatch 2026-02-13T02:21:22.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:22 trial107 ceph-mon[27422]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b545439a-3f32-4b0a-bc6a-17f0d8050c07"}]': finished 2026-02-13T02:21:22.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:22 trial107 ceph-mon[27422]: osd.6 [v2:10.20.193.27:6818/3514139178,v1:10.20.193.27:6819/3514139178] boot 2026-02-13T02:21:22.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:22 trial107 ceph-mon[27422]: osdmap e37: 8 total, 7 up, 8 in 2026-02-13T02:21:22.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:22 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-13T02:21:22.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:22 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-13T02:21:22.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:22 trial107 ceph-mon[27422]: from='client.? 10.20.193.28:0/2381718069' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b545439a-3f32-4b0a-bc6a-17f0d8050c07"} : dispatch 2026-02-13T02:21:22.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:22 trial107 ceph-mon[27422]: from='client.? 10.20.193.28:0/1805716535' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:21:22.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:22 trial107 ceph-mon[27422]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:22.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:22 trial107 ceph-mon[27422]: Cluster is now healthy 2026-02-13T02:21:23.394 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:23 trial028 ceph-mon[26701]: purged_snaps scrub starts 2026-02-13T02:21:23.395 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:23 trial028 ceph-mon[26701]: purged_snaps scrub ok 2026-02-13T02:21:23.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:23 trial027 ceph-mon[24567]: purged_snaps scrub starts 2026-02-13T02:21:23.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:23 trial027 ceph-mon[24567]: purged_snaps scrub ok 2026-02-13T02:21:23.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:23 trial107 ceph-mon[27422]: purged_snaps scrub starts 2026-02-13T02:21:23.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:23 trial107 ceph-mon[27422]: purged_snaps scrub ok 2026-02-13T02:21:24.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:24 trial028 ceph-mon[26701]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-13T02:21:24.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:24 trial028 ceph-mon[26701]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-13T02:21:24.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:24 trial028 ceph-mon[26701]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:24.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:24 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:24.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:24 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:24.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:24 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:24.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:24 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:24.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:24 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:24.495 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:24 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:24.495 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:24 trial028 ceph-mon[26701]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-13T02:21:24.495 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:24 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:24.495 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:24 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:24.495 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:24 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:24.495 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:24 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2026-02-13T02:21:24.495 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:24 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:24.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:24 trial027 ceph-mon[24567]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-13T02:21:24.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:24 trial027 ceph-mon[24567]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-13T02:21:24.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:24 trial027 ceph-mon[24567]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:24.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:24 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:24.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:24 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:24.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:24 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:24.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:24 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:24.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:24 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:24.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:24 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:24.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:24 trial027 ceph-mon[24567]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-13T02:21:24.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:24 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:24.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:24 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:24.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:24 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:24.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:24 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2026-02-13T02:21:24.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:24 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:24.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:24 trial107 ceph-mon[27422]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-13T02:21:24.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:24 trial107 ceph-mon[27422]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-13T02:21:24.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:24 trial107 ceph-mon[27422]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:24.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:24 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:24.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:24 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:24.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:24 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:24.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:24 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:24.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:24 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:24.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:24 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:24.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:24 trial107 ceph-mon[27422]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-13T02:21:24.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:24 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:24.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:24 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:24.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:24 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:24.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:24 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2026-02-13T02:21:24.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:24 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:25.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:25 trial028 ceph-mon[26701]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-13T02:21:25.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:25 trial027 ceph-mon[24567]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-13T02:21:25.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:25 trial107 ceph-mon[27422]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-13T02:21:26.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:26 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:26.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:26 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:26.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:26 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:26.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:26 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:26.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:26 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:26.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:26 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:26.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:26 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:26.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:26 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:26.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:26 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:26.882 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:26.883+0000 7f5728ff96c0 1 -- 10.20.193.28:0/426556166 <== mgr.14116 v2:10.20.193.27:6800/1837452956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7f572c0f14f0 con 0x7f57000b5600 2026-02-13T02:21:26.882 INFO:teuthology.orchestra.run.trial028.stdout:Created osd(s) 7 on host 'trial028' 2026-02-13T02:21:26.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:26.886+0000 7f5731b816c0 1 -- 10.20.193.28:0/426556166 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f57000b5600 msgr2=0x7f57000d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:26.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:26.886+0000 7f5731b816c0 1 --2- 10.20.193.28:0/426556166 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f57000b5600 0x7f57000d5ab0 secure :-1 s=READY pgs=111 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5718005980 tx=0x7f5718005910 comp rx=0 tx=0).stop 2026-02-13T02:21:26.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:26.886+0000 7f5731b816c0 1 -- 10.20.193.28:0/426556166 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f572c110040 msgr2=0x7f572c171380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:26.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:26.886+0000 7f5731b816c0 1 --2- 10.20.193.28:0/426556166 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f572c110040 0x7f572c171380 secure :-1 s=READY pgs=25 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f571c00b6c0 tx=0x7f571c00bb90 comp rx=0 tx=0).stop 2026-02-13T02:21:26.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:26.886+0000 7f5731b816c0 1 -- 10.20.193.28:0/426556166 shutdown_connections 2026-02-13T02:21:26.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:26.886+0000 7f5731b816c0 1 --2- 10.20.193.28:0/426556166 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f57000b5600 0x7f57000d5ab0 unknown :-1 s=CLOSED pgs=111 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:26.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:26.886+0000 7f5731b816c0 1 --2- 10.20.193.28:0/426556166 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f572c1109f0 0x7f572c0f8e40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:26.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:26.886+0000 7f5731b816c0 1 --2- 10.20.193.28:0/426556166 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f572c110040 0x7f572c171380 unknown :-1 s=CLOSED pgs=25 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:26.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:26.886+0000 7f5731b816c0 1 --2- 10.20.193.28:0/426556166 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f572c10eec0 0x7f572c0f85e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:26.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:26.886+0000 7f5731b816c0 1 -- 10.20.193.28:0/426556166 >> 10.20.193.28:0/426556166 conn(0x7f572c0ec5c0 msgr2=0x7f572c0eebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:21:26.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:26.886+0000 7f5731b816c0 1 -- 10.20.193.28:0/426556166 shutdown_connections 2026-02-13T02:21:26.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:26.886+0000 7f5731b816c0 1 -- 10.20.193.28:0/426556166 wait complete. 2026-02-13T02:21:26.928 DEBUG:teuthology.orchestra.run.trial028:osd.7> sudo journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.7.service 2026-02-13T02:21:26.930 INFO:tasks.cephadm:Deploying osd.8 on trial107 with /dev/vg_nvme/lv_2... 2026-02-13T02:21:26.930 DEBUG:teuthology.orchestra.run.trial107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_2 2026-02-13T02:21:27.051 INFO:teuthology.orchestra.run.trial107.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.c/config 2026-02-13T02:21:27.488 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:21:27.507 DEBUG:teuthology.orchestra.run.trial107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph orch daemon add osd trial107:vg_nvme/lv_2 --skip-validation 2026-02-13T02:21:27.627 INFO:teuthology.orchestra.run.trial107.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.c/config 2026-02-13T02:21:27.741 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.743+0000 7fc1b6bb76c0 1 -- 10.20.193.107:0/3646620742 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc1b010e770 msgr2=0x7fc1b016f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:27.742 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.743+0000 7fc1b6bb76c0 1 --2- 10.20.193.107:0/3646620742 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc1b010e770 0x7fc1b016f100 secure :-1 s=READY pgs=26 gs=1 cs=0 l=1 c_cookie=12843780674491207589 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc1ac00aa40 tx=0x7fc1ac02fde0 comp rx=0 tx=0).stop 2026-02-13T02:21:27.742 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.743+0000 7fc1b6bb76c0 1 -- 10.20.193.107:0/3646620742 shutdown_connections 2026-02-13T02:21:27.742 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.743+0000 7fc1b6bb76c0 1 --2- 10.20.193.107:0/3646620742 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc1b010e770 0x7fc1b016f100 unknown :-1 s=CLOSED pgs=26 gs=1 cs=0 l=1 c_cookie=12843780674491207589 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:27.742 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.743+0000 7fc1b6bb76c0 1 --2- 10.20.193.107:0/3646620742 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc1b010ddc0 0x7fc1b010e230 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:27.742 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.743+0000 7fc1b6bb76c0 1 --2- 10.20.193.107:0/3646620742 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc1b010cc40 0x7fc1b010d030 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:27.742 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.743+0000 7fc1b6bb76c0 1 -- 10.20.193.107:0/3646620742 >> 10.20.193.107:0/3646620742 conn(0x7fc1b00ea360 msgr2=0x7fc1b010a770 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:21:27.742 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.743+0000 7fc1b6bb76c0 1 -- 10.20.193.107:0/3646620742 shutdown_connections 2026-02-13T02:21:27.742 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.743+0000 7fc1b6bb76c0 1 -- 10.20.193.107:0/3646620742 wait complete. 2026-02-13T02:21:27.743 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:27 trial107 ceph-mon[27422]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-13T02:21:27.743 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:27 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:27.743 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:27 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:27.743 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:27 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:27.743 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:27 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:27.743 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:27 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:27.743 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:27 trial107 ceph-mon[27422]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-13T02:21:27.743 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:27 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:27.743 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:27 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:27.744 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:27 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:27.744 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:27 trial107 ceph-mon[27422]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2026-02-13T02:21:27.744 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:27 trial107 ceph-mon[27422]: from='osd.7 [v2:10.20.193.28:6816/1405200379,v1:10.20.193.28:6817/1405200379]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2026-02-13T02:21:27.744 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:27 trial107 ceph-mon[27422]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:27.744 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:27 trial107 ceph-mon[27422]: Cluster is now healthy 2026-02-13T02:21:27.744 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:27 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:27.744 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:27 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:27.744 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:27 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:27.747 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:27 trial028 ceph-mon[26701]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-13T02:21:27.747 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:27 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:27.747 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:27 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:27.747 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:27 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:27.748 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:27 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:27.748 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:27 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:27.748 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:27 trial028 ceph-mon[26701]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-13T02:21:27.748 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:27 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:27.748 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:27 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:27.748 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:27 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:27.748 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:27 trial028 ceph-mon[26701]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2026-02-13T02:21:27.748 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:27 trial028 ceph-mon[26701]: from='osd.7 [v2:10.20.193.28:6816/1405200379,v1:10.20.193.28:6817/1405200379]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2026-02-13T02:21:27.748 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:27 trial028 ceph-mon[26701]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:27.748 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:27 trial028 ceph-mon[26701]: Cluster is now healthy 2026-02-13T02:21:27.748 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:27 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:27.748 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:27 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:27.749 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:27 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:27.749 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.743+0000 7fc1b6bb76c0 1 Processor -- start 2026-02-13T02:21:27.749 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.743+0000 7fc1b6bb76c0 1 -- start start 2026-02-13T02:21:27.749 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.744+0000 7fc1b6bb76c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc1b010cc40 0x7fc1b00f6390 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:27.749 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.744+0000 7fc1b6bb76c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc1b010ddc0 0x7fc1b016f100 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:27.749 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.744+0000 7fc1b6bb76c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc1b010e770 0x7fc1b00f6bf0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:27.749 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.744+0000 7fc1b6bb76c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b00f7820 con 0x7fc1b010e770 2026-02-13T02:21:27.749 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.744+0000 7fc1b6bb76c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b00f7990 con 0x7fc1b010ddc0 2026-02-13T02:21:27.750 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.744+0000 7fc1b6bb76c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b00f7b50 con 0x7fc1b010cc40 2026-02-13T02:21:27.750 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.744+0000 7fc1b49286c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc1b010cc40 0x7fc1b00f6390 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:27.750 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.744+0000 7fc1b49286c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc1b010cc40 0x7fc1b00f6390 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.107:3300/0 says I am v2:10.20.193.107:53406/0 (socket says 10.20.193.107:53406) 2026-02-13T02:21:27.750 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.744+0000 7fc1b49286c0 1 -- 10.20.193.107:0/1652883486 learned_addr learned my addr 10.20.193.107:0/1652883486 (peer_addr_for_me v2:10.20.193.107:0/0) 2026-02-13T02:21:27.750 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.744+0000 7fc1a7fff6c0 1 --2- 10.20.193.107:0/1652883486 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc1b010ddc0 0x7fc1b016f100 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:27.750 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.744+0000 7fc1b51296c0 1 --2- 10.20.193.107:0/1652883486 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc1b010e770 0x7fc1b00f6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:27.750 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.744+0000 7fc1b49286c0 1 -- 10.20.193.107:0/1652883486 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc1b010ddc0 msgr2=0x7fc1b016f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:27.750 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.744+0000 7fc1b49286c0 1 --2- 10.20.193.107:0/1652883486 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc1b010ddc0 0x7fc1b016f100 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:27.750 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.744+0000 7fc1b49286c0 1 -- 10.20.193.107:0/1652883486 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc1b010e770 msgr2=0x7fc1b00f6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:27.750 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.744+0000 7fc1b49286c0 1 --2- 10.20.193.107:0/1652883486 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc1b010e770 0x7fc1b00f6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:27.750 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.744+0000 7fc1b49286c0 1 -- 10.20.193.107:0/1652883486 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1ac00a660 con 0x7fc1b010cc40 2026-02-13T02:21:27.750 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.744+0000 7fc1b49286c0 1 --2- 10.20.193.107:0/1652883486 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc1b010cc40 0x7fc1b00f6390 secure :-1 s=READY pgs=27 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc1a0009490 tx=0x7fc1a000bca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:21:27.751 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.744+0000 7fc1a5ffb6c0 1 -- 10.20.193.107:0/1652883486 <== mon.1 v2:10.20.193.107:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc1a0004a80 con 0x7fc1b010cc40 2026-02-13T02:21:27.751 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.744+0000 7fc1b6bb76c0 1 -- 10.20.193.107:0/1652883486 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b00f7e30 con 0x7fc1b010cc40 2026-02-13T02:21:27.751 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.744+0000 7fc1a5ffb6c0 1 -- 10.20.193.107:0/1652883486 <== mon.1 v2:10.20.193.107:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc1a0009030 con 0x7fc1b010cc40 2026-02-13T02:21:27.751 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.744+0000 7fc1a5ffb6c0 1 -- 10.20.193.107:0/1652883486 <== mon.1 v2:10.20.193.107:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc1a000ce30 con 0x7fc1b010cc40 2026-02-13T02:21:27.751 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.744+0000 7fc1b6bb76c0 1 -- 10.20.193.107:0/1652883486 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b00f8360 con 0x7fc1b010cc40 2026-02-13T02:21:27.751 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.745+0000 7fc1b6bb76c0 1 -- 10.20.193.107:0/1652883486 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1b0172fc0 con 0x7fc1b010cc40 2026-02-13T02:21:27.751 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.746+0000 7fc1a5ffb6c0 1 -- 10.20.193.107:0/1652883486 <== mon.1 v2:10.20.193.107:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7fc1a0004060 con 0x7fc1b010cc40 2026-02-13T02:21:27.751 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.746+0000 7fc1a5ffb6c0 1 --2- 10.20.193.107:0/1652883486 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc18c0b5600 0x7fc18c0d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:27.751 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.747+0000 7fc1a5ffb6c0 1 -- 10.20.193.107:0/1652883486 <== mon.1 v2:10.20.193.107:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4878+0+0 (secure 0 0 0) 0x7fc1a00e2e80 con 0x7fc1b010cc40 2026-02-13T02:21:27.751 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.747+0000 7fc1a7fff6c0 1 --2- 10.20.193.107:0/1652883486 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc18c0b5600 0x7fc18c0d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:27.751 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.747+0000 7fc1a5ffb6c0 1 -- 10.20.193.107:0/1652883486 <== mon.1 v2:10.20.193.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fc1a00a6550 con 0x7fc1b010cc40 2026-02-13T02:21:27.751 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.747+0000 7fc1a7fff6c0 1 --2- 10.20.193.107:0/1652883486 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc18c0b5600 0x7fc18c0d5ab0 secure :-1 s=READY pgs=116 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc198005980 tx=0x7fc1980094b0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:21:27.844 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:27.846+0000 7fc1b6bb76c0 1 -- 10.20.193.107:0/1652883486 --> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "trial107:vg_nvme/lv_2", "skip_validation": true, "target": ["mon-mgr", ""]}) -- 0x7fc1b00ed010 con 0x7fc18c0b5600 2026-02-13T02:21:27.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:27 trial027 ceph-mon[24567]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-13T02:21:27.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:27 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:27.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:27 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:27.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:27 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:27.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:27 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:27.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:27 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:27.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:27 trial027 ceph-mon[24567]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-13T02:21:27.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:27 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:27.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:27 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:27.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:27 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:27.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:27 trial027 ceph-mon[24567]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2026-02-13T02:21:27.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:27 trial027 ceph-mon[24567]: from='osd.7 [v2:10.20.193.28:6816/1405200379,v1:10.20.193.28:6817/1405200379]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2026-02-13T02:21:27.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:27 trial027 ceph-mon[24567]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:27.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:27 trial027 ceph-mon[24567]: Cluster is now healthy 2026-02-13T02:21:27.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:27 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:27.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:27 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:27.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:27 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:28.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:28 trial028 ceph-mon[26701]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-13T02:21:28.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:28 trial028 ceph-mon[26701]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2026-02-13T02:21:28.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:28 trial028 ceph-mon[26701]: osdmap e38: 8 total, 7 up, 8 in 2026-02-13T02:21:28.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:28 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-13T02:21:28.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:28 trial028 ceph-mon[26701]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-13T02:21:28.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:28 trial028 ceph-mon[26701]: from='osd.7 [v2:10.20.193.28:6816/1405200379,v1:10.20.193.28:6817/1405200379]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-13T02:21:28.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:28 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:28.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:28 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:28.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:28 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:28.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:28 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:28.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:28 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:28.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:28 trial028 ceph-mon[26701]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:28.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:28 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:28.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:28 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:28.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:28 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:28.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:28 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:28.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:28 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:28.746 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:28 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:28.746 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:28 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:28.746 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:28 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:28.746 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:28 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:28.746 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:28 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:28.746 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:28 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:28.746 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:28 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:28.746 INFO:journalctl@ceph.osd.7.trial028.stdout:Feb 13 02:21:28 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-7[44087]: 2026-02-13T02:21:28.475+0000 7f11021c36c0 -1 osd.7 0 waiting for initial osdmap 2026-02-13T02:21:28.746 INFO:journalctl@ceph.osd.7.trial028.stdout:Feb 13 02:21:28 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-7[44087]: 2026-02-13T02:21:28.480+0000 7f10fe0876c0 -1 osd.7 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-13T02:21:28.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:28 trial107 ceph-mon[27422]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-13T02:21:28.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:28 trial107 ceph-mon[27422]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2026-02-13T02:21:28.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:28 trial107 ceph-mon[27422]: osdmap e38: 8 total, 7 up, 8 in 2026-02-13T02:21:28.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:28 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-13T02:21:28.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:28 trial107 ceph-mon[27422]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-13T02:21:28.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:28 trial107 ceph-mon[27422]: from='osd.7 [v2:10.20.193.28:6816/1405200379,v1:10.20.193.28:6817/1405200379]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-13T02:21:28.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:28 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:28.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:28 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:28.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:28 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:28.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:28 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:28.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:28 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:28.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:28 trial107 ceph-mon[27422]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:28.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:28 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:28.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:28 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:28.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:28 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:28.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:28 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:28.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:28 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:28.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:28 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:28.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:28 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:28.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:28 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:28.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:28 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:28.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:28 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:28.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:28 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:28.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:28 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:28.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:28 trial027 ceph-mon[24567]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-13T02:21:28.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:28 trial027 ceph-mon[24567]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2026-02-13T02:21:28.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:28 trial027 ceph-mon[24567]: osdmap e38: 8 total, 7 up, 8 in 2026-02-13T02:21:28.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:28 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-13T02:21:28.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:28 trial027 ceph-mon[24567]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-13T02:21:28.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:28 trial027 ceph-mon[24567]: from='osd.7 [v2:10.20.193.28:6816/1405200379,v1:10.20.193.28:6817/1405200379]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-13T02:21:28.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:28 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:28.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:28 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:28.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:28 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:28.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:28 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:28.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:28 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:28.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:28 trial027 ceph-mon[24567]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:28.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:28 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:28.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:28 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:28.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:28 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:28.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:28 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:28.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:28 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:28.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:28 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:28.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:28 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:28.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:28 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:28.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:28 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:28.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:28 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:28.911 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:28 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:28.911 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:28 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:29.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:29 trial028 ceph-mon[26701]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial107:vg_nvme/lv_2", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:21:29.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:29 trial028 ceph-mon[26701]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-13T02:21:29.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:29 trial028 ceph-mon[26701]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.6836, "args": ["host=trial028", "root=default"]}]': finished 2026-02-13T02:21:29.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:29 trial028 ceph-mon[26701]: osdmap e39: 8 total, 7 up, 8 in 2026-02-13T02:21:29.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:29 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-13T02:21:29.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:29 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-13T02:21:29.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:29 trial028 ceph-mon[26701]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a7c6bbba-760c-484a-8171-44c70f7d9c89"} : dispatch 2026-02-13T02:21:29.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:29 trial028 ceph-mon[26701]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7c6bbba-760c-484a-8171-44c70f7d9c89"}]': finished 2026-02-13T02:21:29.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:29 trial028 ceph-mon[26701]: osd.7 [v2:10.20.193.28:6816/1405200379,v1:10.20.193.28:6817/1405200379] boot 2026-02-13T02:21:29.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:29 trial028 ceph-mon[26701]: osdmap e40: 9 total, 8 up, 9 in 2026-02-13T02:21:29.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:29 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-13T02:21:29.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:29 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-13T02:21:29.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:29 trial028 ceph-mon[26701]: from='client.? 10.20.193.107:0/4224041769' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a7c6bbba-760c-484a-8171-44c70f7d9c89"} : dispatch 2026-02-13T02:21:29.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:29 trial028 ceph-mon[26701]: from='client.? 10.20.193.107:0/1268297080' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:21:29.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:29 trial028 ceph-mon[26701]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:29.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:29 trial028 ceph-mon[26701]: Cluster is now healthy 2026-02-13T02:21:29.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:29 trial107 ceph-mon[27422]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial107:vg_nvme/lv_2", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:21:29.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:29 trial107 ceph-mon[27422]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-13T02:21:29.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:29 trial107 ceph-mon[27422]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.6836, "args": ["host=trial028", "root=default"]}]': finished 2026-02-13T02:21:29.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:29 trial107 ceph-mon[27422]: osdmap e39: 8 total, 7 up, 8 in 2026-02-13T02:21:29.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:29 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-13T02:21:29.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:29 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-13T02:21:29.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:29 trial107 ceph-mon[27422]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a7c6bbba-760c-484a-8171-44c70f7d9c89"} : dispatch 2026-02-13T02:21:29.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:29 trial107 ceph-mon[27422]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7c6bbba-760c-484a-8171-44c70f7d9c89"}]': finished 2026-02-13T02:21:29.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:29 trial107 ceph-mon[27422]: osd.7 [v2:10.20.193.28:6816/1405200379,v1:10.20.193.28:6817/1405200379] boot 2026-02-13T02:21:29.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:29 trial107 ceph-mon[27422]: osdmap e40: 9 total, 8 up, 9 in 2026-02-13T02:21:29.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:29 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-13T02:21:29.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:29 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-13T02:21:29.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:29 trial107 ceph-mon[27422]: from='client.? 10.20.193.107:0/4224041769' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a7c6bbba-760c-484a-8171-44c70f7d9c89"} : dispatch 2026-02-13T02:21:29.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:29 trial107 ceph-mon[27422]: from='client.? 10.20.193.107:0/1268297080' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:21:29.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:29 trial107 ceph-mon[27422]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:29.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:29 trial107 ceph-mon[27422]: Cluster is now healthy 2026-02-13T02:21:29.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:29 trial027 ceph-mon[24567]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial107:vg_nvme/lv_2", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:21:29.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:29 trial027 ceph-mon[24567]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-13T02:21:29.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:29 trial027 ceph-mon[24567]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.6836, "args": ["host=trial028", "root=default"]}]': finished 2026-02-13T02:21:29.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:29 trial027 ceph-mon[24567]: osdmap e39: 8 total, 7 up, 8 in 2026-02-13T02:21:29.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:29 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-13T02:21:29.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:29 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-13T02:21:29.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:29 trial027 ceph-mon[24567]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a7c6bbba-760c-484a-8171-44c70f7d9c89"} : dispatch 2026-02-13T02:21:29.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:29 trial027 ceph-mon[24567]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7c6bbba-760c-484a-8171-44c70f7d9c89"}]': finished 2026-02-13T02:21:29.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:29 trial027 ceph-mon[24567]: osd.7 [v2:10.20.193.28:6816/1405200379,v1:10.20.193.28:6817/1405200379] boot 2026-02-13T02:21:29.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:29 trial027 ceph-mon[24567]: osdmap e40: 9 total, 8 up, 9 in 2026-02-13T02:21:29.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:29 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-13T02:21:29.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:29 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-13T02:21:29.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:29 trial027 ceph-mon[24567]: from='client.? 10.20.193.107:0/4224041769' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a7c6bbba-760c-484a-8171-44c70f7d9c89"} : dispatch 2026-02-13T02:21:29.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:29 trial027 ceph-mon[24567]: from='client.? 10.20.193.107:0/1268297080' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:21:29.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:29 trial027 ceph-mon[24567]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:29.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:29 trial027 ceph-mon[24567]: Cluster is now healthy 2026-02-13T02:21:30.922 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:30 trial107 ceph-mon[27422]: purged_snaps scrub starts 2026-02-13T02:21:30.922 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:30 trial107 ceph-mon[27422]: purged_snaps scrub ok 2026-02-13T02:21:30.922 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:30 trial107 ceph-mon[27422]: osdmap e41: 9 total, 8 up, 9 in 2026-02-13T02:21:30.922 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:30 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-13T02:21:30.922 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:30 trial107 ceph-mon[27422]: pgmap v110: 1 pgs: 1 remapped; 577 KiB data, 214 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-13T02:21:30.922 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:30 trial107 ceph-mon[27422]: pgmap v111: 1 pgs: 1 remapped; 577 KiB data, 214 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-13T02:21:30.922 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:30 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:30.922 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:30 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:30.923 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:30 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:30.923 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:30 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:30.923 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:30 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:30.923 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:30 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:30.923 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:30 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:30.923 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:30 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:30.923 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:30 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:30.993 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:30 trial028 ceph-mon[26701]: purged_snaps scrub starts 2026-02-13T02:21:30.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:30 trial028 ceph-mon[26701]: purged_snaps scrub ok 2026-02-13T02:21:30.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:30 trial028 ceph-mon[26701]: osdmap e41: 9 total, 8 up, 9 in 2026-02-13T02:21:30.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:30 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-13T02:21:30.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:30 trial028 ceph-mon[26701]: pgmap v110: 1 pgs: 1 remapped; 577 KiB data, 214 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-13T02:21:30.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:30 trial028 ceph-mon[26701]: pgmap v111: 1 pgs: 1 remapped; 577 KiB data, 214 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-13T02:21:30.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:30 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:30.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:30 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:30.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:30 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:30.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:30 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:30.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:30 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:30.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:30 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:30.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:30 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:30.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:30 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:30.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:30 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:31.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:30 trial027 ceph-mon[24567]: purged_snaps scrub starts 2026-02-13T02:21:31.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:30 trial027 ceph-mon[24567]: purged_snaps scrub ok 2026-02-13T02:21:31.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:30 trial027 ceph-mon[24567]: osdmap e41: 9 total, 8 up, 9 in 2026-02-13T02:21:31.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:30 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-13T02:21:31.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:30 trial027 ceph-mon[24567]: pgmap v110: 1 pgs: 1 remapped; 577 KiB data, 214 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-13T02:21:31.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:30 trial027 ceph-mon[24567]: pgmap v111: 1 pgs: 1 remapped; 577 KiB data, 214 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-13T02:21:31.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:30 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:31.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:30 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:31.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:30 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:31.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:30 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:31.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:30 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:31.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:30 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:31.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:30 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:31.160 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:30 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:31.160 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:30 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:31.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:31 trial107 ceph-mon[27422]: pgmap v112: 1 pgs: 1 remapped; 577 KiB data, 214 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-13T02:21:31.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:31 trial107 ceph-mon[27422]: osdmap e42: 9 total, 8 up, 9 in 2026-02-13T02:21:31.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:31 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-13T02:21:31.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:31 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2026-02-13T02:21:31.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:31 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:31.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:31 trial107 ceph-mon[27422]: osdmap e43: 9 total, 8 up, 9 in 2026-02-13T02:21:31.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:31 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-13T02:21:31.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:31 trial107 ceph-mon[27422]: pgmap v115: 1 pgs: 1 remapped; 577 KiB data, 214 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-13T02:21:31.993 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:31 trial028 ceph-mon[26701]: pgmap v112: 1 pgs: 1 remapped; 577 KiB data, 214 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-13T02:21:31.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:31 trial028 ceph-mon[26701]: osdmap e42: 9 total, 8 up, 9 in 2026-02-13T02:21:31.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:31 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-13T02:21:31.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:31 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2026-02-13T02:21:31.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:31 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:31.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:31 trial028 ceph-mon[26701]: osdmap e43: 9 total, 8 up, 9 in 2026-02-13T02:21:31.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:31 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-13T02:21:31.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:31 trial028 ceph-mon[26701]: pgmap v115: 1 pgs: 1 remapped; 577 KiB data, 214 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-13T02:21:32.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:31 trial027 ceph-mon[24567]: pgmap v112: 1 pgs: 1 remapped; 577 KiB data, 214 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-13T02:21:32.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:31 trial027 ceph-mon[24567]: osdmap e42: 9 total, 8 up, 9 in 2026-02-13T02:21:32.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:31 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-13T02:21:32.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:31 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2026-02-13T02:21:32.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:31 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:32.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:31 trial027 ceph-mon[24567]: osdmap e43: 9 total, 8 up, 9 in 2026-02-13T02:21:32.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:31 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-13T02:21:32.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:31 trial027 ceph-mon[24567]: pgmap v115: 1 pgs: 1 remapped; 577 KiB data, 214 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-13T02:21:32.922 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:32 trial107 ceph-mon[27422]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:32.923 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:32 trial107 ceph-mon[27422]: osdmap e44: 9 total, 8 up, 9 in 2026-02-13T02:21:32.923 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:32 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-13T02:21:32.993 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:32 trial028 ceph-mon[26701]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:32.993 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:32 trial028 ceph-mon[26701]: osdmap e44: 9 total, 8 up, 9 in 2026-02-13T02:21:32.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:32 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-13T02:21:33.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:32 trial027 ceph-mon[24567]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:33.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:32 trial027 ceph-mon[24567]: osdmap e44: 9 total, 8 up, 9 in 2026-02-13T02:21:33.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:32 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-13T02:21:33.993 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:33 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:33.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:33 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:33.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:33 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:33.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:33 trial028 ceph-mon[26701]: pgmap v117: 1 pgs: 1 remapped; 577 KiB data, 214 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-13T02:21:34.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:33 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:34.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:33 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:34.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:33 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:34.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:33 trial107 ceph-mon[27422]: pgmap v117: 1 pgs: 1 remapped; 577 KiB data, 214 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-13T02:21:34.117 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:34.119+0000 7fc1a5ffb6c0 1 -- 10.20.193.107:0/1652883486 <== mgr.14116 v2:10.20.193.27:6800/1837452956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7fc1b00ed010 con 0x7fc18c0b5600 2026-02-13T02:21:34.117 INFO:teuthology.orchestra.run.trial107.stdout:Created osd(s) 8 on host 'trial107' 2026-02-13T02:21:34.120 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:34.122+0000 7fc1b6bb76c0 1 -- 10.20.193.107:0/1652883486 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc18c0b5600 msgr2=0x7fc18c0d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:34.120 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:34.122+0000 7fc1b6bb76c0 1 --2- 10.20.193.107:0/1652883486 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc18c0b5600 0x7fc18c0d5ab0 secure :-1 s=READY pgs=116 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc198005980 tx=0x7fc1980094b0 comp rx=0 tx=0).stop 2026-02-13T02:21:34.120 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:34.122+0000 7fc1b6bb76c0 1 -- 10.20.193.107:0/1652883486 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc1b010cc40 msgr2=0x7fc1b00f6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:34.120 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:34.122+0000 7fc1b6bb76c0 1 --2- 10.20.193.107:0/1652883486 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc1b010cc40 0x7fc1b00f6390 secure :-1 s=READY pgs=27 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc1a0009490 tx=0x7fc1a000bca0 comp rx=0 tx=0).stop 2026-02-13T02:21:34.120 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:34.122+0000 7fc1b6bb76c0 1 -- 10.20.193.107:0/1652883486 shutdown_connections 2026-02-13T02:21:34.120 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:34.122+0000 7fc1b6bb76c0 1 --2- 10.20.193.107:0/1652883486 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc18c0b5600 0x7fc18c0d5ab0 unknown :-1 s=CLOSED pgs=116 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:34.120 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:34.122+0000 7fc1b6bb76c0 1 --2- 10.20.193.107:0/1652883486 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc1b010e770 0x7fc1b00f6bf0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:34.120 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:34.122+0000 7fc1b6bb76c0 1 --2- 10.20.193.107:0/1652883486 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc1b010ddc0 0x7fc1b016f100 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:34.120 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:34.122+0000 7fc1b6bb76c0 1 --2- 10.20.193.107:0/1652883486 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc1b010cc40 0x7fc1b00f6390 unknown :-1 s=CLOSED pgs=27 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:34.121 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:34.122+0000 7fc1b6bb76c0 1 -- 10.20.193.107:0/1652883486 >> 10.20.193.107:0/1652883486 conn(0x7fc1b00ea360 msgr2=0x7fc1b00ec900 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:21:34.121 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:34.122+0000 7fc1b6bb76c0 1 -- 10.20.193.107:0/1652883486 shutdown_connections 2026-02-13T02:21:34.121 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:34.122+0000 7fc1b6bb76c0 1 -- 10.20.193.107:0/1652883486 wait complete. 2026-02-13T02:21:34.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:33 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:34.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:33 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:34.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:33 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:34.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:33 trial027 ceph-mon[24567]: pgmap v117: 1 pgs: 1 remapped; 577 KiB data, 214 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-13T02:21:34.160 DEBUG:teuthology.orchestra.run.trial107:osd.8> sudo journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.8.service 2026-02-13T02:21:34.163 INFO:tasks.cephadm:Deploying osd.9 on trial027 with /dev/vg_nvme/lv_1... 2026-02-13T02:21:34.163 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_1 2026-02-13T02:21:34.286 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:21:34.734 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:21:34.752 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph orch daemon add osd trial027:vg_nvme/lv_1 --skip-validation 2026-02-13T02:21:34.872 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:21:34.988 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:34 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:34.988 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:34 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:34.988 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:34 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:34.988 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:34 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:34.988 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:34 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:34.988 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:34 trial027 ceph-mon[24567]: pgmap v118: 1 pgs: 1 remapped; 577 KiB data, 214 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-13T02:21:34.988 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:34 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:34.988 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:34 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:34.989 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:34 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:34.989 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:34 trial027 ceph-mon[24567]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2026-02-13T02:21:34.989 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:34 trial027 ceph-mon[24567]: from='osd.8 [v2:10.20.193.107:6816/1331562104,v1:10.20.193.107:6817/1331562104]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2026-02-13T02:21:34.989 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:34 trial027 ceph-mon[24567]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:34.989 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:34 trial027 ceph-mon[24567]: Cluster is now healthy 2026-02-13T02:21:34.989 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:34 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:34.989 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:34 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:34.989 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:34 trial027 ceph-mon[24567]: pgmap v119: 1 pgs: 1 remapped; 577 KiB data, 214 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-13T02:21:34.989 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:34 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:34.993 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.988+0000 7ff54fdd46c0 1 -- 10.20.193.27:0/1346700379 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff54810eec0 msgr2=0x7ff54810f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:34.993 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.988+0000 7ff54fdd46c0 1 --2- 10.20.193.27:0/1346700379 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff54810eec0 0x7ff54810f2b0 secure :-1 s=READY pgs=171 gs=1 cs=0 l=1 c_cookie=4530776207235146142 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff538009970 tx=0x7ff53802ed20 comp rx=0 tx=0).stop 2026-02-13T02:21:34.993 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.988+0000 7ff54fdd46c0 1 -- 10.20.193.27:0/1346700379 shutdown_connections 2026-02-13T02:21:34.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.988+0000 7ff54fdd46c0 1 --2- 10.20.193.27:0/1346700379 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff5481109f0 0x7ff548171380 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:34.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.988+0000 7ff54fdd46c0 1 --2- 10.20.193.27:0/1346700379 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff548110040 0x7ff5481104b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:34.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.988+0000 7ff54fdd46c0 1 --2- 10.20.193.27:0/1346700379 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff54810eec0 0x7ff54810f2b0 unknown :-1 s=CLOSED pgs=171 gs=1 cs=0 l=1 c_cookie=4530776207235146142 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:34.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.988+0000 7ff54fdd46c0 1 -- 10.20.193.27:0/1346700379 >> 10.20.193.27:0/1346700379 conn(0x7ff5480ec5c0 msgr2=0x7ff54810c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:21:34.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.988+0000 7ff54fdd46c0 1 -- 10.20.193.27:0/1346700379 shutdown_connections 2026-02-13T02:21:34.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.988+0000 7ff54fdd46c0 1 -- 10.20.193.27:0/1346700379 wait complete. 2026-02-13T02:21:34.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.989+0000 7ff54fdd46c0 1 Processor -- start 2026-02-13T02:21:34.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.989+0000 7ff54fdd46c0 1 -- start start 2026-02-13T02:21:34.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.989+0000 7ff54fdd46c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff54810eec0 0x7ff5480f8730 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:34.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.989+0000 7ff54fdd46c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff548110040 0x7ff548171380 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:34.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.989+0000 7ff54fdd46c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff5481109f0 0x7ff5480f8f90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:34.995 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.989+0000 7ff54fdd46c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff5480f9b70 con 0x7ff54810eec0 2026-02-13T02:21:34.995 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.989+0000 7ff54fdd46c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff5480f9ce0 con 0x7ff548110040 2026-02-13T02:21:34.995 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.989+0000 7ff54fdd46c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff5480f9ea0 con 0x7ff5481109f0 2026-02-13T02:21:34.995 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.989+0000 7ff54db496c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff54810eec0 0x7ff5480f8730 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:34.995 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.989+0000 7ff54db496c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff54810eec0 0x7ff5480f8730 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:53944/0 (socket says 10.20.193.27:53944) 2026-02-13T02:21:34.995 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.989+0000 7ff54db496c0 1 -- 10.20.193.27:0/2664675949 learned_addr learned my addr 10.20.193.27:0/2664675949 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:21:34.995 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.989+0000 7ff54d3486c0 1 --2- 10.20.193.27:0/2664675949 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff548110040 0x7ff548171380 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:34.995 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.989+0000 7ff54e34a6c0 1 --2- 10.20.193.27:0/2664675949 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff5481109f0 0x7ff5480f8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:34.995 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.989+0000 7ff54db496c0 1 -- 10.20.193.27:0/2664675949 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff5481109f0 msgr2=0x7ff5480f8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:34.995 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.989+0000 7ff54db496c0 1 --2- 10.20.193.27:0/2664675949 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff5481109f0 0x7ff5480f8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:34.995 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.989+0000 7ff54db496c0 1 -- 10.20.193.27:0/2664675949 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff548110040 msgr2=0x7ff548171380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:34.996 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.989+0000 7ff54db496c0 1 --2- 10.20.193.27:0/2664675949 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff548110040 0x7ff548171380 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:34.996 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.989+0000 7ff54db496c0 1 -- 10.20.193.27:0/2664675949 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff538009590 con 0x7ff54810eec0 2026-02-13T02:21:34.996 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.989+0000 7ff54db496c0 1 --2- 10.20.193.27:0/2664675949 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff54810eec0 0x7ff5480f8730 secure :-1 s=READY pgs=172 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff5380092b0 tx=0x7ff5380046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:21:34.996 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.989+0000 7ff536ffd6c0 1 -- 10.20.193.27:0/2664675949 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff538002b60 con 0x7ff54810eec0 2026-02-13T02:21:34.996 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.989+0000 7ff54fdd46c0 1 -- 10.20.193.27:0/2664675949 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5480fa120 con 0x7ff54810eec0 2026-02-13T02:21:34.996 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.989+0000 7ff536ffd6c0 1 -- 10.20.193.27:0/2664675949 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff538007680 con 0x7ff54810eec0 2026-02-13T02:21:34.996 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.989+0000 7ff536ffd6c0 1 -- 10.20.193.27:0/2664675949 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff5380375d0 con 0x7ff54810eec0 2026-02-13T02:21:34.996 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.989+0000 7ff54fdd46c0 1 -- 10.20.193.27:0/2664675949 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5480fa5f0 con 0x7ff54810eec0 2026-02-13T02:21:34.996 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.990+0000 7ff54fdd46c0 1 -- 10.20.193.27:0/2664675949 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff548175260 con 0x7ff54810eec0 2026-02-13T02:21:34.996 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.991+0000 7ff536ffd6c0 1 -- 10.20.193.27:0/2664675949 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7ff538007cd0 con 0x7ff54810eec0 2026-02-13T02:21:34.997 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:34 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:34.997 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:34 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:34.997 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:34 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:34.997 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:34 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:34.997 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:34 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:34.998 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:34 trial028 ceph-mon[26701]: pgmap v118: 1 pgs: 1 remapped; 577 KiB data, 214 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-13T02:21:34.998 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:34 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:34.998 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:34 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:34.998 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:34 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:34.998 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:34 trial028 ceph-mon[26701]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2026-02-13T02:21:34.998 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:34 trial028 ceph-mon[26701]: from='osd.8 [v2:10.20.193.107:6816/1331562104,v1:10.20.193.107:6817/1331562104]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2026-02-13T02:21:34.998 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:34 trial028 ceph-mon[26701]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:34.998 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:34 trial028 ceph-mon[26701]: Cluster is now healthy 2026-02-13T02:21:34.998 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:34 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:34.998 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:34 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:34.998 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:34 trial028 ceph-mon[26701]: pgmap v119: 1 pgs: 1 remapped; 577 KiB data, 214 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-13T02:21:34.998 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:34 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:34.999 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.991+0000 7ff536ffd6c0 1 --2- 10.20.193.27:0/2664675949 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff5240b5600 0x7ff5240d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:34.999 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.992+0000 7ff536ffd6c0 1 -- 10.20.193.27:0/2664675949 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5310+0+0 (secure 0 0 0) 0x7ff538107270 con 0x7ff54810eec0 2026-02-13T02:21:34.999 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.992+0000 7ff54d3486c0 1 --2- 10.20.193.27:0/2664675949 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff5240b5600 0x7ff5240d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:34.999 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.992+0000 7ff536ffd6c0 1 -- 10.20.193.27:0/2664675949 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7ff5380ca820 con 0x7ff54810eec0 2026-02-13T02:21:34.999 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:34.992+0000 7ff54d3486c0 1 --2- 10.20.193.27:0/2664675949 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff5240b5600 0x7ff5240d5ab0 secure :-1 s=READY pgs=121 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff53c005980 tx=0x7ff53c005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:21:35.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:34 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:35.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:34 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:35.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:34 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:35.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:34 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:35.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:34 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:35.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:34 trial107 ceph-mon[27422]: pgmap v118: 1 pgs: 1 remapped; 577 KiB data, 214 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-13T02:21:35.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:34 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:35.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:34 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:35.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:34 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:35.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:34 trial107 ceph-mon[27422]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2026-02-13T02:21:35.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:34 trial107 ceph-mon[27422]: from='osd.8 [v2:10.20.193.107:6816/1331562104,v1:10.20.193.107:6817/1331562104]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2026-02-13T02:21:35.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:34 trial107 ceph-mon[27422]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:35.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:34 trial107 ceph-mon[27422]: Cluster is now healthy 2026-02-13T02:21:35.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:34 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:35.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:34 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:35.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:34 trial107 ceph-mon[27422]: pgmap v119: 1 pgs: 1 remapped; 577 KiB data, 214 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-13T02:21:35.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:34 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:35.092 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:35.090+0000 7ff54fdd46c0 1 -- 10.20.193.27:0/2664675949 --> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "trial027:vg_nvme/lv_1", "skip_validation": true, "target": ["mon-mgr", ""]}) -- 0x7ff54804ca20 con 0x7ff5240b5600 2026-02-13T02:21:35.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:35 trial027 ceph-mon[24567]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2026-02-13T02:21:35.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:35 trial027 ceph-mon[24567]: osdmap e45: 9 total, 8 up, 9 in 2026-02-13T02:21:35.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:35 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-13T02:21:35.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:35 trial027 ceph-mon[24567]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.6836, "args": ["host=trial107", "root=default"]} : dispatch 2026-02-13T02:21:35.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:35 trial027 ceph-mon[24567]: from='osd.8 [v2:10.20.193.107:6816/1331562104,v1:10.20.193.107:6817/1331562104]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.6836, "args": ["host=trial107", "root=default"]} : dispatch 2026-02-13T02:21:35.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:35 trial027 ceph-mon[24567]: from='client.14565 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial027:vg_nvme/lv_1", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:21:35.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:35 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:35.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:35 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:35.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:35 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:35.911 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:35 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:35.911 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:35 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:35.911 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:35 trial027 ceph-mon[24567]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:35.911 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:35 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:35.911 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:35 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:35.911 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:35 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:35.912 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:35 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:35.912 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:35 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:35.912 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:35 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:35.912 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:35 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:35.912 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:35 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:35.912 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:35 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:35.913 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:35 trial027 ceph-mon[24567]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-13T02:21:35.913 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:35 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:35.913 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:35 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:35.913 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:35 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:36.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:35 trial107 ceph-mon[27422]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2026-02-13T02:21:36.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:35 trial107 ceph-mon[27422]: osdmap e45: 9 total, 8 up, 9 in 2026-02-13T02:21:36.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:35 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-13T02:21:36.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:35 trial107 ceph-mon[27422]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.6836, "args": ["host=trial107", "root=default"]} : dispatch 2026-02-13T02:21:36.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:35 trial107 ceph-mon[27422]: from='osd.8 [v2:10.20.193.107:6816/1331562104,v1:10.20.193.107:6817/1331562104]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.6836, "args": ["host=trial107", "root=default"]} : dispatch 2026-02-13T02:21:36.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:35 trial107 ceph-mon[27422]: from='client.14565 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial027:vg_nvme/lv_1", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:21:36.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:35 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:36.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:35 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:36.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:35 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:36.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:35 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:36.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:35 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:36.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:35 trial107 ceph-mon[27422]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:36.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:35 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:36.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:35 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:36.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:35 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:36.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:35 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:36.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:35 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:36.003 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:35 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:36.003 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:35 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:36.003 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:35 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:36.003 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:35 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:36.003 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:35 trial107 ceph-mon[27422]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-13T02:21:36.003 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:35 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:36.003 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:35 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:36.003 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:35 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:36.004 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:21:35 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:21:35.751+0000 7f97d7b6e6c0 -1 osd.8 0 waiting for initial osdmap 2026-02-13T02:21:36.004 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:21:35 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:21:35.756+0000 7f97d3a326c0 -1 osd.8 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-13T02:21:36.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:35 trial028 ceph-mon[26701]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2026-02-13T02:21:36.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:35 trial028 ceph-mon[26701]: osdmap e45: 9 total, 8 up, 9 in 2026-02-13T02:21:36.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:35 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-13T02:21:36.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:35 trial028 ceph-mon[26701]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.6836, "args": ["host=trial107", "root=default"]} : dispatch 2026-02-13T02:21:36.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:35 trial028 ceph-mon[26701]: from='osd.8 [v2:10.20.193.107:6816/1331562104,v1:10.20.193.107:6817/1331562104]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.6836, "args": ["host=trial107", "root=default"]} : dispatch 2026-02-13T02:21:36.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:35 trial028 ceph-mon[26701]: from='client.14565 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial027:vg_nvme/lv_1", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:21:36.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:35 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:36.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:35 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:36.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:35 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:36.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:35 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:36.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:35 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:36.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:35 trial028 ceph-mon[26701]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:36.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:35 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:36.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:35 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:36.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:35 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:36.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:35 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:36.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:35 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:36.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:35 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:36.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:35 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:36.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:35 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:36.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:35 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:36.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:35 trial028 ceph-mon[26701]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-13T02:21:36.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:35 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:36.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:35 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:36.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:35 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:37.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:36 trial107 ceph-mon[27422]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.6836, "args": ["host=trial107", "root=default"]}]': finished 2026-02-13T02:21:37.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:36 trial107 ceph-mon[27422]: osdmap e46: 9 total, 8 up, 9 in 2026-02-13T02:21:37.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:36 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-13T02:21:37.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:36 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-13T02:21:37.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:36 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/444220821' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "72cbd422-3c3a-4904-9075-f543671d4f48"} : dispatch 2026-02-13T02:21:37.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:36 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/444220821' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "72cbd422-3c3a-4904-9075-f543671d4f48"}]': finished 2026-02-13T02:21:37.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:36 trial107 ceph-mon[27422]: osd.8 [v2:10.20.193.107:6816/1331562104,v1:10.20.193.107:6817/1331562104] boot 2026-02-13T02:21:37.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:36 trial107 ceph-mon[27422]: osdmap e47: 10 total, 9 up, 10 in 2026-02-13T02:21:37.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:36 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-13T02:21:37.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:36 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-13T02:21:37.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:36 trial107 ceph-mon[27422]: osdmap e48: 10 total, 9 up, 10 in 2026-02-13T02:21:37.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:36 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-13T02:21:37.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:36 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1679019922' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:21:37.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:36 trial107 ceph-mon[27422]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:37.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:36 trial107 ceph-mon[27422]: Cluster is now healthy 2026-02-13T02:21:37.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:36 trial027 ceph-mon[24567]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.6836, "args": ["host=trial107", "root=default"]}]': finished 2026-02-13T02:21:37.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:36 trial027 ceph-mon[24567]: osdmap e46: 9 total, 8 up, 9 in 2026-02-13T02:21:37.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:36 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-13T02:21:37.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:36 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-13T02:21:37.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:36 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/444220821' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "72cbd422-3c3a-4904-9075-f543671d4f48"} : dispatch 2026-02-13T02:21:37.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:36 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/444220821' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "72cbd422-3c3a-4904-9075-f543671d4f48"}]': finished 2026-02-13T02:21:37.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:36 trial027 ceph-mon[24567]: osd.8 [v2:10.20.193.107:6816/1331562104,v1:10.20.193.107:6817/1331562104] boot 2026-02-13T02:21:37.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:36 trial027 ceph-mon[24567]: osdmap e47: 10 total, 9 up, 10 in 2026-02-13T02:21:37.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:36 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-13T02:21:37.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:36 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-13T02:21:37.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:36 trial027 ceph-mon[24567]: osdmap e48: 10 total, 9 up, 10 in 2026-02-13T02:21:37.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:36 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-13T02:21:37.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:36 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1679019922' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:21:37.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:36 trial027 ceph-mon[24567]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:37.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:36 trial027 ceph-mon[24567]: Cluster is now healthy 2026-02-13T02:21:37.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:36 trial028 ceph-mon[26701]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.6836, "args": ["host=trial107", "root=default"]}]': finished 2026-02-13T02:21:37.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:36 trial028 ceph-mon[26701]: osdmap e46: 9 total, 8 up, 9 in 2026-02-13T02:21:37.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:36 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-13T02:21:37.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:36 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-13T02:21:37.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:36 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/444220821' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "72cbd422-3c3a-4904-9075-f543671d4f48"} : dispatch 2026-02-13T02:21:37.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:36 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/444220821' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "72cbd422-3c3a-4904-9075-f543671d4f48"}]': finished 2026-02-13T02:21:37.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:36 trial028 ceph-mon[26701]: osd.8 [v2:10.20.193.107:6816/1331562104,v1:10.20.193.107:6817/1331562104] boot 2026-02-13T02:21:37.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:36 trial028 ceph-mon[26701]: osdmap e47: 10 total, 9 up, 10 in 2026-02-13T02:21:37.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:36 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-13T02:21:37.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:36 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-13T02:21:37.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:36 trial028 ceph-mon[26701]: osdmap e48: 10 total, 9 up, 10 in 2026-02-13T02:21:37.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:36 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-13T02:21:37.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:36 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1679019922' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:21:37.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:36 trial028 ceph-mon[26701]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:37.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:36 trial028 ceph-mon[26701]: Cluster is now healthy 2026-02-13T02:21:37.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:37 trial027 ceph-mon[24567]: purged_snaps scrub starts 2026-02-13T02:21:37.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:37 trial027 ceph-mon[24567]: purged_snaps scrub ok 2026-02-13T02:21:37.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:37 trial027 ceph-mon[24567]: osdmap e49: 10 total, 9 up, 10 in 2026-02-13T02:21:37.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:37 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-13T02:21:37.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:37 trial027 ceph-mon[24567]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 6.2 TiB / 6.2 TiB avail 2026-02-13T02:21:38.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:37 trial107 ceph-mon[27422]: purged_snaps scrub starts 2026-02-13T02:21:38.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:37 trial107 ceph-mon[27422]: purged_snaps scrub ok 2026-02-13T02:21:38.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:37 trial107 ceph-mon[27422]: osdmap e49: 10 total, 9 up, 10 in 2026-02-13T02:21:38.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:37 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-13T02:21:38.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:37 trial107 ceph-mon[27422]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 6.2 TiB / 6.2 TiB avail 2026-02-13T02:21:38.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:37 trial028 ceph-mon[26701]: purged_snaps scrub starts 2026-02-13T02:21:38.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:37 trial028 ceph-mon[26701]: purged_snaps scrub ok 2026-02-13T02:21:38.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:37 trial028 ceph-mon[26701]: osdmap e49: 10 total, 9 up, 10 in 2026-02-13T02:21:38.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:37 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-13T02:21:38.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:37 trial028 ceph-mon[26701]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 6.2 TiB / 6.2 TiB avail 2026-02-13T02:21:39.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:38 trial107 ceph-mon[27422]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 6.2 TiB / 6.2 TiB avail 2026-02-13T02:21:39.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:38 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:39.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:38 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:39.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:38 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:39.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:38 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:39.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:38 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:39.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:38 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:39.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:38 trial107 ceph-mon[27422]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 6.2 TiB / 6.2 TiB avail 2026-02-13T02:21:39.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:38 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:39.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:38 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:39.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:38 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:39.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:38 trial107 ceph-mon[27422]: osdmap e50: 10 total, 9 up, 10 in 2026-02-13T02:21:39.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:38 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-13T02:21:39.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:38 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2026-02-13T02:21:39.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:38 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:39.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:38 trial107 ceph-mon[27422]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 6.2 TiB / 6.2 TiB avail 2026-02-13T02:21:39.058 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:38 trial027 ceph-mon[24567]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 6.2 TiB / 6.2 TiB avail 2026-02-13T02:21:39.059 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:38 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:39.059 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:38 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:39.059 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:38 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:39.059 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:38 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:39.059 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:38 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:39.059 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:38 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:39.059 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:38 trial027 ceph-mon[24567]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 6.2 TiB / 6.2 TiB avail 2026-02-13T02:21:39.059 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:38 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:39.059 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:38 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:39.059 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:38 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:39.060 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:38 trial027 ceph-mon[24567]: osdmap e50: 10 total, 9 up, 10 in 2026-02-13T02:21:39.060 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:38 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-13T02:21:39.060 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:38 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2026-02-13T02:21:39.060 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:38 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:39.060 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:38 trial027 ceph-mon[24567]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 6.2 TiB / 6.2 TiB avail 2026-02-13T02:21:39.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:38 trial028 ceph-mon[26701]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 6.2 TiB / 6.2 TiB avail 2026-02-13T02:21:39.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:38 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:39.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:38 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:39.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:38 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:39.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:38 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:39.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:38 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:39.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:38 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:39.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:38 trial028 ceph-mon[26701]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 6.2 TiB / 6.2 TiB avail 2026-02-13T02:21:39.245 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:38 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:39.245 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:38 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:39.245 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:38 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:39.245 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:38 trial028 ceph-mon[26701]: osdmap e50: 10 total, 9 up, 10 in 2026-02-13T02:21:39.245 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:38 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-13T02:21:39.245 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:38 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2026-02-13T02:21:39.245 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:38 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:39.245 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:38 trial028 ceph-mon[26701]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 6.2 TiB / 6.2 TiB avail 2026-02-13T02:21:40.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:39 trial107 ceph-mon[27422]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:40.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:39 trial107 ceph-mon[27422]: osdmap e51: 10 total, 9 up, 10 in 2026-02-13T02:21:40.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:39 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-13T02:21:40.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:39 trial027 ceph-mon[24567]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:40.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:39 trial027 ceph-mon[24567]: osdmap e51: 10 total, 9 up, 10 in 2026-02-13T02:21:40.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:39 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-13T02:21:40.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:39 trial028 ceph-mon[26701]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:40.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:39 trial028 ceph-mon[26701]: osdmap e51: 10 total, 9 up, 10 in 2026-02-13T02:21:40.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:39 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-13T02:21:41.032 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:40 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:41.033 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:40 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:41.033 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:40 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:41.033 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:40 trial027 ceph-mon[24567]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 6.2 TiB / 6.2 TiB avail; 178 KiB/s, 0 objects/s recovering 2026-02-13T02:21:41.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:40 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:41.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:40 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:41.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:40 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:41.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:40 trial028 ceph-mon[26701]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 6.2 TiB / 6.2 TiB avail; 178 KiB/s, 0 objects/s recovering 2026-02-13T02:21:41.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:40 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:41.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:40 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:41.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:40 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:41.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:40 trial107 ceph-mon[27422]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 6.2 TiB / 6.2 TiB avail; 178 KiB/s, 0 objects/s recovering 2026-02-13T02:21:41.564 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:41.562+0000 7ff536ffd6c0 1 -- 10.20.193.27:0/2664675949 <== mgr.14116 v2:10.20.193.27:6800/1837452956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7ff54804ca20 con 0x7ff5240b5600 2026-02-13T02:21:41.565 INFO:teuthology.orchestra.run.trial027.stdout:Created osd(s) 9 on host 'trial027' 2026-02-13T02:21:41.567 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:41.564+0000 7ff54fdd46c0 1 -- 10.20.193.27:0/2664675949 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff5240b5600 msgr2=0x7ff5240d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:41.567 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:41.564+0000 7ff54fdd46c0 1 --2- 10.20.193.27:0/2664675949 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff5240b5600 0x7ff5240d5ab0 secure :-1 s=READY pgs=121 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff53c005980 tx=0x7ff53c005910 comp rx=0 tx=0).stop 2026-02-13T02:21:41.567 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:41.564+0000 7ff54fdd46c0 1 -- 10.20.193.27:0/2664675949 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff54810eec0 msgr2=0x7ff5480f8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:41.567 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:41.564+0000 7ff54fdd46c0 1 --2- 10.20.193.27:0/2664675949 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff54810eec0 0x7ff5480f8730 secure :-1 s=READY pgs=172 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff5380092b0 tx=0x7ff5380046e0 comp rx=0 tx=0).stop 2026-02-13T02:21:41.568 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:41.564+0000 7ff54fdd46c0 1 -- 10.20.193.27:0/2664675949 shutdown_connections 2026-02-13T02:21:41.568 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:41.564+0000 7ff54fdd46c0 1 --2- 10.20.193.27:0/2664675949 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff5240b5600 0x7ff5240d5ab0 unknown :-1 s=CLOSED pgs=121 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:41.568 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:41.564+0000 7ff54fdd46c0 1 --2- 10.20.193.27:0/2664675949 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff5481109f0 0x7ff5480f8f90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:41.568 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:41.564+0000 7ff54fdd46c0 1 --2- 10.20.193.27:0/2664675949 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff548110040 0x7ff548171380 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:41.568 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:41.564+0000 7ff54fdd46c0 1 --2- 10.20.193.27:0/2664675949 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff54810eec0 0x7ff5480f8730 unknown :-1 s=CLOSED pgs=172 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:41.568 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:41.564+0000 7ff54fdd46c0 1 -- 10.20.193.27:0/2664675949 >> 10.20.193.27:0/2664675949 conn(0x7ff5480ec5c0 msgr2=0x7ff5480eebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:21:41.568 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:41.564+0000 7ff54fdd46c0 1 -- 10.20.193.27:0/2664675949 shutdown_connections 2026-02-13T02:21:41.568 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:41.564+0000 7ff54fdd46c0 1 -- 10.20.193.27:0/2664675949 wait complete. 2026-02-13T02:21:41.608 DEBUG:teuthology.orchestra.run.trial027:osd.9> sudo journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.9.service 2026-02-13T02:21:41.611 INFO:tasks.cephadm:Deploying osd.10 on trial028 with /dev/vg_nvme/lv_1... 2026-02-13T02:21:41.611 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_1 2026-02-13T02:21:41.731 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.b/config 2026-02-13T02:21:42.186 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-13T02:21:42.203 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph orch daemon add osd trial028:vg_nvme/lv_1 --skip-validation 2026-02-13T02:21:42.322 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.b/config 2026-02-13T02:21:42.437 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:42 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:42.437 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:42 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:42.437 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:42 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:42.437 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:42 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:42.437 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:42 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:42.437 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:42 trial028 ceph-mon[26701]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 6.2 TiB / 6.2 TiB avail; 172 KiB/s, 0 objects/s recovering 2026-02-13T02:21:42.437 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:42 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:42.438 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:42 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:42.438 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:42 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:42.438 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:42 trial028 ceph-mon[26701]: from='osd.9 [v2:10.20.193.27:6826/1444927853,v1:10.20.193.27:6827/1444927853]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2026-02-13T02:21:42.438 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:42 trial028 ceph-mon[26701]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:42.438 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:42 trial028 ceph-mon[26701]: Cluster is now healthy 2026-02-13T02:21:42.438 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:42 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:42.438 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:42 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:42.438 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:42 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:42.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.438+0000 7f14e46cb6c0 1 -- 10.20.193.28:0/1646623923 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f14dc110040 msgr2=0x7f14dc1104b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:42.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.438+0000 7f14e46cb6c0 1 --2- 10.20.193.28:0/1646623923 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f14dc110040 0x7f14dc1104b0 secure :-1 s=READY pgs=33 gs=1 cs=0 l=1 c_cookie=11548671240185227298 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f14d0009970 tx=0x7f14d002ed50 comp rx=0 tx=0).stop 2026-02-13T02:21:42.442 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.438+0000 7f14e46cb6c0 1 -- 10.20.193.28:0/1646623923 shutdown_connections 2026-02-13T02:21:42.442 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.438+0000 7f14e46cb6c0 1 --2- 10.20.193.28:0/1646623923 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f14dc1109f0 0x7f14dc171380 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:42.442 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.438+0000 7f14e46cb6c0 1 --2- 10.20.193.28:0/1646623923 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f14dc110040 0x7f14dc1104b0 unknown :-1 s=CLOSED pgs=33 gs=1 cs=0 l=1 c_cookie=11548671240185227298 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:42.442 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.438+0000 7f14e46cb6c0 1 --2- 10.20.193.28:0/1646623923 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f14dc10eec0 0x7f14dc10f2b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:42.442 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.438+0000 7f14e46cb6c0 1 -- 10.20.193.28:0/1646623923 >> 10.20.193.28:0/1646623923 conn(0x7f14dc0ec5c0 msgr2=0x7f14dc10c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:21:42.442 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.439+0000 7f14e46cb6c0 1 -- 10.20.193.28:0/1646623923 shutdown_connections 2026-02-13T02:21:42.442 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.439+0000 7f14e46cb6c0 1 -- 10.20.193.28:0/1646623923 wait complete. 2026-02-13T02:21:42.442 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.439+0000 7f14e46cb6c0 1 Processor -- start 2026-02-13T02:21:42.442 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.439+0000 7f14e46cb6c0 1 -- start start 2026-02-13T02:21:42.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.439+0000 7f14e46cb6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f14dc10eec0 0x7f14dc0f8700 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:42.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.439+0000 7f14e46cb6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f14dc110040 0x7f14dc171380 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:42.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.439+0000 7f14e46cb6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f14dc1109f0 0x7f14dc0f8f60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:42.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.439+0000 7f14e46cb6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc0f9b90 con 0x7f14dc110040 2026-02-13T02:21:42.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.439+0000 7f14e46cb6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc0f9d00 con 0x7f14dc1109f0 2026-02-13T02:21:42.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.439+0000 7f14e46cb6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc0f9ec0 con 0x7f14dc10eec0 2026-02-13T02:21:42.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.439+0000 7f14e2c3d6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f14dc1109f0 0x7f14dc0f8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:42.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.439+0000 7f14e2c3d6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f14dc1109f0 0x7f14dc0f8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:41790/0 (socket says 10.20.193.28:41790) 2026-02-13T02:21:42.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.439+0000 7f14e2c3d6c0 1 -- 10.20.193.28:0/2778822536 learned_addr learned my addr 10.20.193.28:0/2778822536 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-13T02:21:42.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.439+0000 7f14e1c3b6c0 1 --2- 10.20.193.28:0/2778822536 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f14dc110040 0x7f14dc171380 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:42.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.439+0000 7f14e243c6c0 1 --2- 10.20.193.28:0/2778822536 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f14dc10eec0 0x7f14dc0f8700 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:42.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.439+0000 7f14e2c3d6c0 1 -- 10.20.193.28:0/2778822536 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f14dc10eec0 msgr2=0x7f14dc0f8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:42.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.439+0000 7f14e2c3d6c0 1 --2- 10.20.193.28:0/2778822536 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f14dc10eec0 0x7f14dc0f8700 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:42.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.439+0000 7f14e2c3d6c0 1 -- 10.20.193.28:0/2778822536 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f14dc110040 msgr2=0x7f14dc171380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:42.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.439+0000 7f14e2c3d6c0 1 --2- 10.20.193.28:0/2778822536 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f14dc110040 0x7f14dc171380 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:42.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.439+0000 7f14e2c3d6c0 1 -- 10.20.193.28:0/2778822536 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d0009590 con 0x7f14dc1109f0 2026-02-13T02:21:42.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.439+0000 7f14e2c3d6c0 1 --2- 10.20.193.28:0/2778822536 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f14dc1109f0 0x7f14dc0f8f60 secure :-1 s=READY pgs=34 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f14d8006ea0 tx=0x7f14d800cca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:21:42.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.439+0000 7f14c37fe6c0 1 -- 10.20.193.28:0/2778822536 <== mon.2 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f14d8003a30 con 0x7f14dc1109f0 2026-02-13T02:21:42.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.440+0000 7f14c37fe6c0 1 -- 10.20.193.28:0/2778822536 <== mon.2 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f14d80054e0 con 0x7f14dc1109f0 2026-02-13T02:21:42.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.440+0000 7f14c37fe6c0 1 -- 10.20.193.28:0/2778822536 <== mon.2 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f14d800dc30 con 0x7f14dc1109f0 2026-02-13T02:21:42.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.440+0000 7f14e46cb6c0 1 -- 10.20.193.28:0/2778822536 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14dc0fa1a0 con 0x7f14dc1109f0 2026-02-13T02:21:42.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.440+0000 7f14e46cb6c0 1 -- 10.20.193.28:0/2778822536 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14dc0fa630 con 0x7f14dc1109f0 2026-02-13T02:21:42.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.440+0000 7f14e46cb6c0 1 -- 10.20.193.28:0/2778822536 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14dc1752e0 con 0x7f14dc1109f0 2026-02-13T02:21:42.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.441+0000 7f14c37fe6c0 1 -- 10.20.193.28:0/2778822536 <== mon.2 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7f14d8005080 con 0x7f14dc1109f0 2026-02-13T02:21:42.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.442+0000 7f14c37fe6c0 1 --2- 10.20.193.28:0/2778822536 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f14b80b5600 0x7f14b80d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:42.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.442+0000 7f14e243c6c0 1 --2- 10.20.193.28:0/2778822536 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f14b80b5600 0x7f14b80d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:42.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.442+0000 7f14c37fe6c0 1 -- 10.20.193.28:0/2778822536 <== mon.2 v2:10.20.193.28:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5742+0+0 (secure 0 0 0) 0x7f14d80e4230 con 0x7f14dc1109f0 2026-02-13T02:21:42.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.442+0000 7f14c37fe6c0 1 -- 10.20.193.28:0/2778822536 <== mon.2 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f14d80a75a0 con 0x7f14dc1109f0 2026-02-13T02:21:42.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.442+0000 7f14e243c6c0 1 --2- 10.20.193.28:0/2778822536 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f14b80b5600 0x7f14b80d5ab0 secure :-1 s=READY pgs=126 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f14cc006b60 tx=0x7f14cc006af0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:21:42.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:42 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:42.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:42 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:42.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:42 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:42.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:42 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:42.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:42 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:42.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:42 trial107 ceph-mon[27422]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 6.2 TiB / 6.2 TiB avail; 172 KiB/s, 0 objects/s recovering 2026-02-13T02:21:42.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:42 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:42.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:42 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:42.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:42 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:42.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:42 trial107 ceph-mon[27422]: from='osd.9 [v2:10.20.193.27:6826/1444927853,v1:10.20.193.27:6827/1444927853]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2026-02-13T02:21:42.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:42 trial107 ceph-mon[27422]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:42.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:42 trial107 ceph-mon[27422]: Cluster is now healthy 2026-02-13T02:21:42.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:42 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:42.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:42 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:42.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:42 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:42.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:42.543+0000 7f14e46cb6c0 1 -- 10.20.193.28:0/2778822536 --> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_1", "skip_validation": true, "target": ["mon-mgr", ""]}) -- 0x7f14dc0ef2f0 con 0x7f14b80b5600 2026-02-13T02:21:42.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:42 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:42.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:42 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:42.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:42 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:42.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:42 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:42.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:42 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:42.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:42 trial027 ceph-mon[24567]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 6.2 TiB / 6.2 TiB avail; 172 KiB/s, 0 objects/s recovering 2026-02-13T02:21:42.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:42 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:42.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:42 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:42.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:42 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:42.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:42 trial027 ceph-mon[24567]: from='osd.9 [v2:10.20.193.27:6826/1444927853,v1:10.20.193.27:6827/1444927853]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2026-02-13T02:21:42.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:42 trial027 ceph-mon[24567]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:42.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:42 trial027 ceph-mon[24567]: Cluster is now healthy 2026-02-13T02:21:42.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:42 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:42.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:42 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:42.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:42 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:43.449 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:43 trial028 ceph-mon[26701]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 6.2 TiB / 6.2 TiB avail; 147 KiB/s, 0 objects/s recovering 2026-02-13T02:21:43.449 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:43 trial028 ceph-mon[26701]: from='osd.9 [v2:10.20.193.27:6826/1444927853,v1:10.20.193.27:6827/1444927853]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2026-02-13T02:21:43.449 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:43 trial028 ceph-mon[26701]: osdmap e52: 10 total, 9 up, 10 in 2026-02-13T02:21:43.449 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:43 trial028 ceph-mon[26701]: from='osd.9 [v2:10.20.193.27:6826/1444927853,v1:10.20.193.27:6827/1444927853]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.6836, "args": ["host=trial027", "root=default"]} : dispatch 2026-02-13T02:21:43.449 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:43 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-13T02:21:43.450 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:43 trial028 ceph-mon[26701]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_1", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:21:43.450 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:43 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:43.450 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:43 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:43.450 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:43 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:43.450 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:43 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:43.450 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:43 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:43.450 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:43 trial028 ceph-mon[26701]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:43.450 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:43 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:43.450 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:43 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:43.451 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:43 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:43.451 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:43 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:43.451 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:43 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:43.451 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:43 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:43.451 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:43 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:43.451 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:43 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:43.451 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:43 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:43.451 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:43 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:43.451 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:43 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:43.451 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:43 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:43.451 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:43 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:43.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:43 trial107 ceph-mon[27422]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 6.2 TiB / 6.2 TiB avail; 147 KiB/s, 0 objects/s recovering 2026-02-13T02:21:43.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:43 trial107 ceph-mon[27422]: from='osd.9 [v2:10.20.193.27:6826/1444927853,v1:10.20.193.27:6827/1444927853]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2026-02-13T02:21:43.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:43 trial107 ceph-mon[27422]: osdmap e52: 10 total, 9 up, 10 in 2026-02-13T02:21:43.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:43 trial107 ceph-mon[27422]: from='osd.9 [v2:10.20.193.27:6826/1444927853,v1:10.20.193.27:6827/1444927853]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.6836, "args": ["host=trial027", "root=default"]} : dispatch 2026-02-13T02:21:43.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:43 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-13T02:21:43.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:43 trial107 ceph-mon[27422]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_1", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:21:43.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:43 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:43.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:43 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:43.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:43 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:43.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:43 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:43.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:43 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:43.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:43 trial107 ceph-mon[27422]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:43.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:43 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:43.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:43 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:43.502 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:43 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:43.502 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:43 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:43.502 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:43 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:43.502 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:43 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:43.502 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:43 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:43.502 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:43 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:43.502 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:43 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:43.502 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:43 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:43.502 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:43 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:43.502 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:43 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:43.502 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:43 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:43.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:43 trial027 ceph-mon[24567]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 6.2 TiB / 6.2 TiB avail; 147 KiB/s, 0 objects/s recovering 2026-02-13T02:21:43.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:43 trial027 ceph-mon[24567]: from='osd.9 [v2:10.20.193.27:6826/1444927853,v1:10.20.193.27:6827/1444927853]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2026-02-13T02:21:43.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:43 trial027 ceph-mon[24567]: osdmap e52: 10 total, 9 up, 10 in 2026-02-13T02:21:43.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:43 trial027 ceph-mon[24567]: from='osd.9 [v2:10.20.193.27:6826/1444927853,v1:10.20.193.27:6827/1444927853]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.6836, "args": ["host=trial027", "root=default"]} : dispatch 2026-02-13T02:21:43.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:43 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-13T02:21:43.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:43 trial027 ceph-mon[24567]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_1", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:21:43.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:43 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:43.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:43 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:43.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:43 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:43.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:43 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:43.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:43 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:43.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:43 trial027 ceph-mon[24567]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:43.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:43 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:43.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:43 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:43.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:43 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:43.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:43 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:43.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:43 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:43.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:43 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:43.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:43 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:43.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:43 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:43.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:43 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:43.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:43 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:43.661 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:43 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:43.661 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:43 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:43.661 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:43 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:43.661 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:21:43 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:21:43.202+0000 7f2e27a376c0 -1 osd.9 0 waiting for initial osdmap 2026-02-13T02:21:43.661 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:21:43 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:21:43.209+0000 7f2e238fb6c0 -1 osd.9 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-13T02:21:44.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:44 trial028 ceph-mon[26701]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 6.2 TiB / 6.2 TiB avail; 137 KiB/s, 0 objects/s recovering 2026-02-13T02:21:44.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:44 trial028 ceph-mon[26701]: from='osd.9 [v2:10.20.193.27:6826/1444927853,v1:10.20.193.27:6827/1444927853]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.6836, "args": ["host=trial027", "root=default"]}]': finished 2026-02-13T02:21:44.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:44 trial028 ceph-mon[26701]: osdmap e53: 10 total, 9 up, 10 in 2026-02-13T02:21:44.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:44 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-13T02:21:44.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:44 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-13T02:21:44.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:44 trial028 ceph-mon[26701]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4bb7b75e-adf7-4c38-9343-dba513b57ed7"} : dispatch 2026-02-13T02:21:44.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:44 trial028 ceph-mon[26701]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4bb7b75e-adf7-4c38-9343-dba513b57ed7"}]': finished 2026-02-13T02:21:44.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:44 trial028 ceph-mon[26701]: osd.9 [v2:10.20.193.27:6826/1444927853,v1:10.20.193.27:6827/1444927853] boot 2026-02-13T02:21:44.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:44 trial028 ceph-mon[26701]: osdmap e54: 11 total, 10 up, 11 in 2026-02-13T02:21:44.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:44 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-13T02:21:44.495 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:44 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-13T02:21:44.495 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:44 trial028 ceph-mon[26701]: from='client.? 10.20.193.28:0/1904154638' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4bb7b75e-adf7-4c38-9343-dba513b57ed7"} : dispatch 2026-02-13T02:21:44.495 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:44 trial028 ceph-mon[26701]: from='client.? 10.20.193.28:0/1088721225' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:21:44.495 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:44 trial028 ceph-mon[26701]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:44.495 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:44 trial028 ceph-mon[26701]: Cluster is now healthy 2026-02-13T02:21:44.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:44 trial107 ceph-mon[27422]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 6.2 TiB / 6.2 TiB avail; 137 KiB/s, 0 objects/s recovering 2026-02-13T02:21:44.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:44 trial107 ceph-mon[27422]: from='osd.9 [v2:10.20.193.27:6826/1444927853,v1:10.20.193.27:6827/1444927853]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.6836, "args": ["host=trial027", "root=default"]}]': finished 2026-02-13T02:21:44.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:44 trial107 ceph-mon[27422]: osdmap e53: 10 total, 9 up, 10 in 2026-02-13T02:21:44.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:44 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-13T02:21:44.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:44 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-13T02:21:44.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:44 trial107 ceph-mon[27422]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4bb7b75e-adf7-4c38-9343-dba513b57ed7"} : dispatch 2026-02-13T02:21:44.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:44 trial107 ceph-mon[27422]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4bb7b75e-adf7-4c38-9343-dba513b57ed7"}]': finished 2026-02-13T02:21:44.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:44 trial107 ceph-mon[27422]: osd.9 [v2:10.20.193.27:6826/1444927853,v1:10.20.193.27:6827/1444927853] boot 2026-02-13T02:21:44.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:44 trial107 ceph-mon[27422]: osdmap e54: 11 total, 10 up, 11 in 2026-02-13T02:21:44.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:44 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-13T02:21:44.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:44 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-13T02:21:44.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:44 trial107 ceph-mon[27422]: from='client.? 10.20.193.28:0/1904154638' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4bb7b75e-adf7-4c38-9343-dba513b57ed7"} : dispatch 2026-02-13T02:21:44.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:44 trial107 ceph-mon[27422]: from='client.? 10.20.193.28:0/1088721225' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:21:44.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:44 trial107 ceph-mon[27422]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:44.502 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:44 trial107 ceph-mon[27422]: Cluster is now healthy 2026-02-13T02:21:44.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:44 trial027 ceph-mon[24567]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 6.2 TiB / 6.2 TiB avail; 137 KiB/s, 0 objects/s recovering 2026-02-13T02:21:44.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:44 trial027 ceph-mon[24567]: from='osd.9 [v2:10.20.193.27:6826/1444927853,v1:10.20.193.27:6827/1444927853]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.6836, "args": ["host=trial027", "root=default"]}]': finished 2026-02-13T02:21:44.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:44 trial027 ceph-mon[24567]: osdmap e53: 10 total, 9 up, 10 in 2026-02-13T02:21:44.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:44 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-13T02:21:44.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:44 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-13T02:21:44.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:44 trial027 ceph-mon[24567]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4bb7b75e-adf7-4c38-9343-dba513b57ed7"} : dispatch 2026-02-13T02:21:44.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:44 trial027 ceph-mon[24567]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4bb7b75e-adf7-4c38-9343-dba513b57ed7"}]': finished 2026-02-13T02:21:44.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:44 trial027 ceph-mon[24567]: osd.9 [v2:10.20.193.27:6826/1444927853,v1:10.20.193.27:6827/1444927853] boot 2026-02-13T02:21:44.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:44 trial027 ceph-mon[24567]: osdmap e54: 11 total, 10 up, 11 in 2026-02-13T02:21:44.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:44 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-13T02:21:44.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:44 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-13T02:21:44.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:44 trial027 ceph-mon[24567]: from='client.? 10.20.193.28:0/1904154638' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4bb7b75e-adf7-4c38-9343-dba513b57ed7"} : dispatch 2026-02-13T02:21:44.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:44 trial027 ceph-mon[24567]: from='client.? 10.20.193.28:0/1088721225' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:21:44.661 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:44 trial027 ceph-mon[24567]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:44.661 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:44 trial027 ceph-mon[24567]: Cluster is now healthy 2026-02-13T02:21:45.608 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:45 trial028 ceph-mon[26701]: purged_snaps scrub starts 2026-02-13T02:21:45.608 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:45 trial028 ceph-mon[26701]: purged_snaps scrub ok 2026-02-13T02:21:45.608 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:45 trial028 ceph-mon[26701]: osdmap e55: 11 total, 10 up, 11 in 2026-02-13T02:21:45.608 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:45 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-13T02:21:45.608 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:45 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:45.608 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:45 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:45.608 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:45 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:45.609 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:45 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:45.609 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:45 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:45.609 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:45 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:45.609 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:45 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:45.609 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:45 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:45.609 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:45 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:45.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:45 trial027 ceph-mon[24567]: purged_snaps scrub starts 2026-02-13T02:21:45.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:45 trial027 ceph-mon[24567]: purged_snaps scrub ok 2026-02-13T02:21:45.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:45 trial027 ceph-mon[24567]: osdmap e55: 11 total, 10 up, 11 in 2026-02-13T02:21:45.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:45 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-13T02:21:45.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:45 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:45.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:45 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:45.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:45 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:45.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:45 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:45.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:45 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:45.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:45 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:45.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:45 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:45.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:45 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:45.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:45 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:45.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:45 trial107 ceph-mon[27422]: purged_snaps scrub starts 2026-02-13T02:21:45.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:45 trial107 ceph-mon[27422]: purged_snaps scrub ok 2026-02-13T02:21:45.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:45 trial107 ceph-mon[27422]: osdmap e55: 11 total, 10 up, 11 in 2026-02-13T02:21:45.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:45 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-13T02:21:45.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:45 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:45.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:45 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:45.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:45 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:45.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:45 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:45.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:45 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:45.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:45 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:45.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:45 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:45.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:45 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:45.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:45 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:46.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:46 trial027 ceph-mon[24567]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-13T02:21:46.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:46 trial027 ceph-mon[24567]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-13T02:21:46.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:46 trial027 ceph-mon[24567]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-13T02:21:46.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:46 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2026-02-13T02:21:46.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:46 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:46.676 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:46 trial028 ceph-mon[26701]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-13T02:21:46.676 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:46 trial028 ceph-mon[26701]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-13T02:21:46.676 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:46 trial028 ceph-mon[26701]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-13T02:21:46.676 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:46 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2026-02-13T02:21:46.676 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:46 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:46.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:46 trial107 ceph-mon[27422]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-13T02:21:46.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:46 trial107 ceph-mon[27422]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-13T02:21:46.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:46 trial107 ceph-mon[27422]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-13T02:21:46.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:46 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2026-02-13T02:21:46.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:46 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:47.643 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:47 trial028 ceph-mon[26701]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-13T02:21:47.643 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:47 trial028 ceph-mon[26701]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:47.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:47 trial027 ceph-mon[24567]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-13T02:21:47.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:47 trial027 ceph-mon[24567]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:47.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:47 trial107 ceph-mon[27422]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-13T02:21:47.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:47 trial107 ceph-mon[27422]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:48.651 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:48 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:48.651 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:48 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:48.651 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:48 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:48.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:48 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:48.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:48 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:48.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:48 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:48.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:48 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:48.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:48 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:48.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:48 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:48.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:48.930+0000 7f14c37fe6c0 1 -- 10.20.193.28:0/2778822536 <== mgr.14116 v2:10.20.193.27:6800/1837452956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f14dc0ef2f0 con 0x7f14b80b5600 2026-02-13T02:21:48.929 INFO:teuthology.orchestra.run.trial028.stdout:Created osd(s) 10 on host 'trial028' 2026-02-13T02:21:48.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:48.933+0000 7f14e46cb6c0 1 -- 10.20.193.28:0/2778822536 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f14b80b5600 msgr2=0x7f14b80d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:48.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:48.933+0000 7f14e46cb6c0 1 --2- 10.20.193.28:0/2778822536 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f14b80b5600 0x7f14b80d5ab0 secure :-1 s=READY pgs=126 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f14cc006b60 tx=0x7f14cc006af0 comp rx=0 tx=0).stop 2026-02-13T02:21:48.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:48.933+0000 7f14e46cb6c0 1 -- 10.20.193.28:0/2778822536 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f14dc1109f0 msgr2=0x7f14dc0f8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:48.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:48.933+0000 7f14e46cb6c0 1 --2- 10.20.193.28:0/2778822536 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f14dc1109f0 0x7f14dc0f8f60 secure :-1 s=READY pgs=34 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f14d8006ea0 tx=0x7f14d800cca0 comp rx=0 tx=0).stop 2026-02-13T02:21:48.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:48.933+0000 7f14e46cb6c0 1 -- 10.20.193.28:0/2778822536 shutdown_connections 2026-02-13T02:21:48.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:48.933+0000 7f14e46cb6c0 1 --2- 10.20.193.28:0/2778822536 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f14b80b5600 0x7f14b80d5ab0 unknown :-1 s=CLOSED pgs=126 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:48.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:48.933+0000 7f14e46cb6c0 1 --2- 10.20.193.28:0/2778822536 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f14dc1109f0 0x7f14dc0f8f60 unknown :-1 s=CLOSED pgs=34 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:48.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:48.933+0000 7f14e46cb6c0 1 --2- 10.20.193.28:0/2778822536 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f14dc110040 0x7f14dc171380 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:48.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:48.933+0000 7f14e46cb6c0 1 --2- 10.20.193.28:0/2778822536 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f14dc10eec0 0x7f14dc0f8700 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:48.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:48.933+0000 7f14e46cb6c0 1 -- 10.20.193.28:0/2778822536 >> 10.20.193.28:0/2778822536 conn(0x7f14dc0ec5c0 msgr2=0x7f14dc0eebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:21:48.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:48.933+0000 7f14e46cb6c0 1 -- 10.20.193.28:0/2778822536 shutdown_connections 2026-02-13T02:21:48.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-13T02:21:48.933+0000 7f14e46cb6c0 1 -- 10.20.193.28:0/2778822536 wait complete. 2026-02-13T02:21:48.972 DEBUG:teuthology.orchestra.run.trial028:osd.10> sudo journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.10.service 2026-02-13T02:21:48.973 INFO:tasks.cephadm:Deploying osd.11 on trial107 with /dev/vg_nvme/lv_1... 2026-02-13T02:21:48.973 DEBUG:teuthology.orchestra.run.trial107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_1 2026-02-13T02:21:49.095 INFO:teuthology.orchestra.run.trial107.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.c/config 2026-02-13T02:21:49.416 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:49 trial028 ceph-mon[26701]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-13T02:21:49.416 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:49 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:49.416 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:49 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:49.416 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:49 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:49.416 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:49 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:49.416 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:49 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:49.416 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:49 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:49.416 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:49 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:49.416 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:49 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:49.416 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:49 trial028 ceph-mon[26701]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:49.416 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:49 trial028 ceph-mon[26701]: Cluster is now healthy 2026-02-13T02:21:49.416 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:49 trial028 ceph-mon[26701]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2026-02-13T02:21:49.416 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:49 trial028 ceph-mon[26701]: from='osd.10 [v2:10.20.193.28:6824/2255755797,v1:10.20.193.28:6825/2255755797]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2026-02-13T02:21:49.416 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:49 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:49.416 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:49 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:49.416 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:49 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:49.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:49 trial107 ceph-mon[27422]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-13T02:21:49.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:49 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:49.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:49 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:49.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:49 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:49.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:49 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:49.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:49 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:49.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:49 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:49.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:49 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:49.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:49 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:49.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:49 trial107 ceph-mon[27422]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:49.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:49 trial107 ceph-mon[27422]: Cluster is now healthy 2026-02-13T02:21:49.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:49 trial107 ceph-mon[27422]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2026-02-13T02:21:49.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:49 trial107 ceph-mon[27422]: from='osd.10 [v2:10.20.193.28:6824/2255755797,v1:10.20.193.28:6825/2255755797]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2026-02-13T02:21:49.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:49 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:49.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:49 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:49.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:49 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:49.535 INFO:teuthology.orchestra.run.trial107.stdout: 2026-02-13T02:21:49.551 DEBUG:teuthology.orchestra.run.trial107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph orch daemon add osd trial107:vg_nvme/lv_1 --skip-validation 2026-02-13T02:21:49.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:49 trial027 ceph-mon[24567]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-13T02:21:49.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:49 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:49.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:49 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:49.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:49 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:49.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:49 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:49.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:49 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:49.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:49 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:49.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:49 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:49.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:49 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:49.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:49 trial027 ceph-mon[24567]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:49.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:49 trial027 ceph-mon[24567]: Cluster is now healthy 2026-02-13T02:21:49.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:49 trial027 ceph-mon[24567]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2026-02-13T02:21:49.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:49 trial027 ceph-mon[24567]: from='osd.10 [v2:10.20.193.28:6824/2255755797,v1:10.20.193.28:6825/2255755797]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2026-02-13T02:21:49.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:49 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:49.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:49 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:49.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:49 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:49.669 INFO:teuthology.orchestra.run.trial107.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.c/config 2026-02-13T02:21:49.797 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.799+0000 7fa4324a96c0 1 -- 10.20.193.107:0/3595457412 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa42c1109f0 msgr2=0x7fa42c171380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:49.797 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.799+0000 7fa4324a96c0 1 --2- 10.20.193.107:0/3595457412 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa42c1109f0 0x7fa42c171380 secure :-1 s=READY pgs=35 gs=1 cs=0 l=1 c_cookie=16297421361216269611 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa42000aa40 tx=0x7fa42002fde0 comp rx=0 tx=0).stop 2026-02-13T02:21:49.797 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.799+0000 7fa4324a96c0 1 -- 10.20.193.107:0/3595457412 shutdown_connections 2026-02-13T02:21:49.798 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.799+0000 7fa4324a96c0 1 --2- 10.20.193.107:0/3595457412 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa42c1109f0 0x7fa42c171380 unknown :-1 s=CLOSED pgs=35 gs=1 cs=0 l=1 c_cookie=16297421361216269611 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:49.798 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.799+0000 7fa4324a96c0 1 --2- 10.20.193.107:0/3595457412 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa42c110040 0x7fa42c1104b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:49.798 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.799+0000 7fa4324a96c0 1 --2- 10.20.193.107:0/3595457412 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa42c10eec0 0x7fa42c10f2b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:49.798 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.799+0000 7fa4324a96c0 1 -- 10.20.193.107:0/3595457412 >> 10.20.193.107:0/3595457412 conn(0x7fa42c0ec5c0 msgr2=0x7fa42c10c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:21:49.798 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.799+0000 7fa4324a96c0 1 -- 10.20.193.107:0/3595457412 shutdown_connections 2026-02-13T02:21:49.798 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.799+0000 7fa4324a96c0 1 -- 10.20.193.107:0/3595457412 wait complete. 2026-02-13T02:21:49.798 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.799+0000 7fa4324a96c0 1 Processor -- start 2026-02-13T02:21:49.798 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.799+0000 7fa4324a96c0 1 -- start start 2026-02-13T02:21:49.798 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.800+0000 7fa4324a96c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa42c10eec0 0x7fa42c0f85e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:49.798 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.800+0000 7fa4324a96c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa42c110040 0x7fa42c171380 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:49.798 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.800+0000 7fa4324a96c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa42c1109f0 0x7fa42c0f8e40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:49.798 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.800+0000 7fa4324a96c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa42c0f9a70 con 0x7fa42c1109f0 2026-02-13T02:21:49.798 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.800+0000 7fa4324a96c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa42c0f9be0 con 0x7fa42c10eec0 2026-02-13T02:21:49.798 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.800+0000 7fa4324a96c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa42c0f9da0 con 0x7fa42c110040 2026-02-13T02:21:49.798 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.800+0000 7fa42b7fe6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa42c110040 0x7fa42c171380 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:49.798 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.800+0000 7fa42b7fe6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa42c110040 0x7fa42c171380 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.107:3300/0 says I am v2:10.20.193.107:51934/0 (socket says 10.20.193.107:51934) 2026-02-13T02:21:49.798 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.800+0000 7fa42b7fe6c0 1 -- 10.20.193.107:0/2551076897 learned_addr learned my addr 10.20.193.107:0/2551076897 (peer_addr_for_me v2:10.20.193.107:0/0) 2026-02-13T02:21:49.798 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.800+0000 7fa42bfff6c0 1 --2- 10.20.193.107:0/2551076897 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa42c10eec0 0x7fa42c0f85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:49.798 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.800+0000 7fa430a1b6c0 1 --2- 10.20.193.107:0/2551076897 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa42c1109f0 0x7fa42c0f8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:49.798 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.800+0000 7fa42b7fe6c0 1 -- 10.20.193.107:0/2551076897 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa42c10eec0 msgr2=0x7fa42c0f85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:49.798 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.800+0000 7fa42b7fe6c0 1 --2- 10.20.193.107:0/2551076897 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa42c10eec0 0x7fa42c0f85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:49.798 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.800+0000 7fa42b7fe6c0 1 -- 10.20.193.107:0/2551076897 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa42c1109f0 msgr2=0x7fa42c0f8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:49.798 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.800+0000 7fa42b7fe6c0 1 --2- 10.20.193.107:0/2551076897 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa42c1109f0 0x7fa42c0f8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:49.798 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.800+0000 7fa42b7fe6c0 1 -- 10.20.193.107:0/2551076897 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42000a660 con 0x7fa42c110040 2026-02-13T02:21:49.798 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.800+0000 7fa42b7fe6c0 1 --2- 10.20.193.107:0/2551076897 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa42c110040 0x7fa42c171380 secure :-1 s=READY pgs=36 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa41c005dc0 tx=0x7fa41c00bbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:21:49.798 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.800+0000 7fa4297fa6c0 1 -- 10.20.193.107:0/2551076897 <== mon.1 v2:10.20.193.107:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa41c004250 con 0x7fa42c110040 2026-02-13T02:21:49.798 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.800+0000 7fa4324a96c0 1 -- 10.20.193.107:0/2551076897 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa42c0fa080 con 0x7fa42c110040 2026-02-13T02:21:49.798 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.800+0000 7fa4297fa6c0 1 -- 10.20.193.107:0/2551076897 <== mon.1 v2:10.20.193.107:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa41c009070 con 0x7fa42c110040 2026-02-13T02:21:49.798 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.800+0000 7fa4297fa6c0 1 -- 10.20.193.107:0/2551076897 <== mon.1 v2:10.20.193.107:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa41c00cd70 con 0x7fa42c110040 2026-02-13T02:21:49.799 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.800+0000 7fa4324a96c0 1 -- 10.20.193.107:0/2551076897 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa42c0fa510 con 0x7fa42c110040 2026-02-13T02:21:49.799 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.800+0000 7fa4324a96c0 1 -- 10.20.193.107:0/2551076897 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa42c175260 con 0x7fa42c110040 2026-02-13T02:21:49.799 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.802+0000 7fa4297fa6c0 1 -- 10.20.193.107:0/2551076897 <== mon.1 v2:10.20.193.107:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7fa41c0028c0 con 0x7fa42c110040 2026-02-13T02:21:49.800 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.802+0000 7fa4297fa6c0 1 --2- 10.20.193.107:0/2551076897 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa4000b5600 0x7fa4000d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:49.800 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.803+0000 7fa4297fa6c0 1 -- 10.20.193.107:0/2551076897 <== mon.1 v2:10.20.193.107:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6153+0+0 (secure 0 0 0) 0x7fa41c0e3390 con 0x7fa42c110040 2026-02-13T02:21:49.800 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.803+0000 7fa42bfff6c0 1 --2- 10.20.193.107:0/2551076897 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa4000b5600 0x7fa4000d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:49.800 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.803+0000 7fa42bfff6c0 1 --2- 10.20.193.107:0/2551076897 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa4000b5600 0x7fa4000d5ab0 secure :-1 s=READY pgs=131 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa418004750 tx=0x7fa418009260 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:21:49.800 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.803+0000 7fa4297fa6c0 1 -- 10.20.193.107:0/2551076897 <== mon.1 v2:10.20.193.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fa41c0a6560 con 0x7fa42c110040 2026-02-13T02:21:49.896 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:49.898+0000 7fa4324a96c0 1 -- 10.20.193.107:0/2551076897 --> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "trial107:vg_nvme/lv_1", "skip_validation": true, "target": ["mon-mgr", ""]}) -- 0x7fa42c04ca20 con 0x7fa4000b5600 2026-02-13T02:21:50.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:50 trial107 ceph-mon[27422]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-13T02:21:50.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:50 trial107 ceph-mon[27422]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-13T02:21:50.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:50 trial107 ceph-mon[27422]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2026-02-13T02:21:50.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:50 trial107 ceph-mon[27422]: osdmap e56: 11 total, 10 up, 11 in 2026-02-13T02:21:50.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:50 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-13T02:21:50.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:50 trial107 ceph-mon[27422]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-13T02:21:50.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:50 trial107 ceph-mon[27422]: from='osd.10 [v2:10.20.193.28:6824/2255755797,v1:10.20.193.28:6825/2255755797]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-13T02:21:50.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:50 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:50.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:50 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:50.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:50 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:50.502 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:50 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:50.502 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:50 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:50.502 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:50 trial107 ceph-mon[27422]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:50.502 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:50 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:50.502 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:50 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:50.502 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:50 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:50.502 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:50 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:50.502 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:50 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:50.502 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:50 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:50.502 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:50 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:50.502 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:50 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:50.503 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:50 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:50.503 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:50 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:50.503 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:50 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:50.503 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:50 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:50.503 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:50 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:50.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:50 trial027 ceph-mon[24567]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-13T02:21:50.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:50 trial027 ceph-mon[24567]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-13T02:21:50.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:50 trial027 ceph-mon[24567]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2026-02-13T02:21:50.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:50 trial027 ceph-mon[24567]: osdmap e56: 11 total, 10 up, 11 in 2026-02-13T02:21:50.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:50 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-13T02:21:50.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:50 trial027 ceph-mon[24567]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-13T02:21:50.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:50 trial027 ceph-mon[24567]: from='osd.10 [v2:10.20.193.28:6824/2255755797,v1:10.20.193.28:6825/2255755797]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-13T02:21:50.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:50 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:50.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:50 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:50.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:50 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:50.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:50 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:50.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:50 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:50.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:50 trial027 ceph-mon[24567]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:50.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:50 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:50.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:50 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:50.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:50 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:50.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:50 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:50.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:50 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:50.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:50 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:50.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:50 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:50.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:50 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:50.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:50 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:50.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:50 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:50.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:50 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:50.661 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:50 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:50.661 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:50 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:50.704 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:50 trial028 ceph-mon[26701]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-13T02:21:50.704 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:50 trial028 ceph-mon[26701]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-13T02:21:50.704 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:50 trial028 ceph-mon[26701]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2026-02-13T02:21:50.704 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:50 trial028 ceph-mon[26701]: osdmap e56: 11 total, 10 up, 11 in 2026-02-13T02:21:50.704 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:50 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-13T02:21:50.704 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:50 trial028 ceph-mon[26701]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-13T02:21:50.705 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:50 trial028 ceph-mon[26701]: from='osd.10 [v2:10.20.193.28:6824/2255755797,v1:10.20.193.28:6825/2255755797]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-13T02:21:50.705 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:50 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:50.705 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:50 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:50.705 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:50 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:50.705 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:50 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:50.705 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:50 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:50.705 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:50 trial028 ceph-mon[26701]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:50.705 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:50 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:50.705 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:50 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:50.705 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:50 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:50.705 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:50 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:50.706 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:50 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:50.706 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:50 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:50.706 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:50 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:50.706 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:50 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:50.706 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:50 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:50.706 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:50 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:50.706 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:50 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:50.706 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:50 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:50.706 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:50 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:50.994 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:21:50 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:21:50.704+0000 7fec4f4446c0 -1 osd.10 0 waiting for initial osdmap 2026-02-13T02:21:50.994 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:21:50 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:21:50.711+0000 7fec473006c0 -1 osd.10 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-13T02:21:51.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:51 trial027 ceph-mon[24567]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial107:vg_nvme/lv_1", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:21:51.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:51 trial027 ceph-mon[24567]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-13T02:21:51.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:51 trial027 ceph-mon[24567]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "115fe8c6-77fc-41dd-ade0-db8eea024974"} : dispatch 2026-02-13T02:21:51.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:51 trial027 ceph-mon[24567]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.6836, "args": ["host=trial028", "root=default"]}]': finished 2026-02-13T02:21:51.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:51 trial027 ceph-mon[24567]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "115fe8c6-77fc-41dd-ade0-db8eea024974"}]': finished 2026-02-13T02:21:51.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:51 trial027 ceph-mon[24567]: osdmap e57: 12 total, 10 up, 12 in 2026-02-13T02:21:51.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:51 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-13T02:21:51.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:51 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-13T02:21:51.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:51 trial027 ceph-mon[24567]: from='client.? 10.20.193.107:0/12855114' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "115fe8c6-77fc-41dd-ade0-db8eea024974"} : dispatch 2026-02-13T02:21:51.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:51 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-13T02:21:51.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:51 trial027 ceph-mon[24567]: from='client.? 10.20.193.107:0/913833856' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:21:51.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:51 trial027 ceph-mon[24567]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:51.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:51 trial027 ceph-mon[24567]: Cluster is now healthy 2026-02-13T02:21:51.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:51 trial027 ceph-mon[24567]: osd.10 [v2:10.20.193.28:6824/2255755797,v1:10.20.193.28:6825/2255755797] boot 2026-02-13T02:21:51.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:51 trial027 ceph-mon[24567]: osdmap e58: 12 total, 11 up, 12 in 2026-02-13T02:21:51.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:51 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-13T02:21:51.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:51 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-13T02:21:51.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:51 trial028 ceph-mon[26701]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial107:vg_nvme/lv_1", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:21:51.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:51 trial028 ceph-mon[26701]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-13T02:21:51.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:51 trial028 ceph-mon[26701]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "115fe8c6-77fc-41dd-ade0-db8eea024974"} : dispatch 2026-02-13T02:21:51.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:51 trial028 ceph-mon[26701]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.6836, "args": ["host=trial028", "root=default"]}]': finished 2026-02-13T02:21:51.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:51 trial028 ceph-mon[26701]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "115fe8c6-77fc-41dd-ade0-db8eea024974"}]': finished 2026-02-13T02:21:51.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:51 trial028 ceph-mon[26701]: osdmap e57: 12 total, 10 up, 12 in 2026-02-13T02:21:51.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:51 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-13T02:21:51.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:51 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-13T02:21:51.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:51 trial028 ceph-mon[26701]: from='client.? 10.20.193.107:0/12855114' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "115fe8c6-77fc-41dd-ade0-db8eea024974"} : dispatch 2026-02-13T02:21:51.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:51 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-13T02:21:51.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:51 trial028 ceph-mon[26701]: from='client.? 10.20.193.107:0/913833856' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:21:51.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:51 trial028 ceph-mon[26701]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:51.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:51 trial028 ceph-mon[26701]: Cluster is now healthy 2026-02-13T02:21:51.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:51 trial028 ceph-mon[26701]: osd.10 [v2:10.20.193.28:6824/2255755797,v1:10.20.193.28:6825/2255755797] boot 2026-02-13T02:21:51.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:51 trial028 ceph-mon[26701]: osdmap e58: 12 total, 11 up, 12 in 2026-02-13T02:21:51.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:51 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-13T02:21:51.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:51 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-13T02:21:51.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:51 trial107 ceph-mon[27422]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial107:vg_nvme/lv_1", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:21:51.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:51 trial107 ceph-mon[27422]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-13T02:21:51.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:51 trial107 ceph-mon[27422]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "115fe8c6-77fc-41dd-ade0-db8eea024974"} : dispatch 2026-02-13T02:21:51.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:51 trial107 ceph-mon[27422]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.6836, "args": ["host=trial028", "root=default"]}]': finished 2026-02-13T02:21:51.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:51 trial107 ceph-mon[27422]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "115fe8c6-77fc-41dd-ade0-db8eea024974"}]': finished 2026-02-13T02:21:51.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:51 trial107 ceph-mon[27422]: osdmap e57: 12 total, 10 up, 12 in 2026-02-13T02:21:51.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:51 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-13T02:21:51.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:51 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-13T02:21:51.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:51 trial107 ceph-mon[27422]: from='client.? 10.20.193.107:0/12855114' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "115fe8c6-77fc-41dd-ade0-db8eea024974"} : dispatch 2026-02-13T02:21:51.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:51 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-13T02:21:51.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:51 trial107 ceph-mon[27422]: from='client.? 10.20.193.107:0/913833856' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-13T02:21:51.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:51 trial107 ceph-mon[27422]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:51.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:51 trial107 ceph-mon[27422]: Cluster is now healthy 2026-02-13T02:21:51.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:51 trial107 ceph-mon[27422]: osd.10 [v2:10.20.193.28:6824/2255755797,v1:10.20.193.28:6825/2255755797] boot 2026-02-13T02:21:51.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:51 trial107 ceph-mon[27422]: osdmap e58: 12 total, 11 up, 12 in 2026-02-13T02:21:51.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:51 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-13T02:21:51.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:51 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-13T02:21:52.615 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:52 trial107 ceph-mon[27422]: purged_snaps scrub starts 2026-02-13T02:21:52.615 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:52 trial107 ceph-mon[27422]: purged_snaps scrub ok 2026-02-13T02:21:52.615 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:52 trial107 ceph-mon[27422]: osdmap e59: 12 total, 11 up, 12 in 2026-02-13T02:21:52.615 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:52 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-13T02:21:52.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:52 trial027 ceph-mon[24567]: purged_snaps scrub starts 2026-02-13T02:21:52.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:52 trial027 ceph-mon[24567]: purged_snaps scrub ok 2026-02-13T02:21:52.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:52 trial027 ceph-mon[24567]: osdmap e59: 12 total, 11 up, 12 in 2026-02-13T02:21:52.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:52 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-13T02:21:52.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:52 trial028 ceph-mon[26701]: purged_snaps scrub starts 2026-02-13T02:21:52.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:52 trial028 ceph-mon[26701]: purged_snaps scrub ok 2026-02-13T02:21:52.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:52 trial028 ceph-mon[26701]: osdmap e59: 12 total, 11 up, 12 in 2026-02-13T02:21:52.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:52 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-13T02:21:53.406 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:53 trial107 ceph-mon[27422]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 7.5 TiB / 7.5 TiB avail 2026-02-13T02:21:53.406 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:53 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:53.406 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:53 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:53.406 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:53 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:53.406 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:53 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:53.406 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:53 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:53.407 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:53 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:53.407 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:53 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:53.407 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:53 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:53.407 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:53 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:53.407 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:53 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2026-02-13T02:21:53.407 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:53 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:53.407 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:53 trial107 ceph-mon[27422]: osdmap e60: 12 total, 11 up, 12 in 2026-02-13T02:21:53.407 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:53 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-13T02:21:53.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:53 trial027 ceph-mon[24567]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 7.5 TiB / 7.5 TiB avail 2026-02-13T02:21:53.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:53 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:53.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:53 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:53.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:53 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:53.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:53 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:53.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:53 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:53.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:53 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:53.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:53 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:53.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:53 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:53.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:53 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:53.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:53 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2026-02-13T02:21:53.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:53 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:53.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:53 trial027 ceph-mon[24567]: osdmap e60: 12 total, 11 up, 12 in 2026-02-13T02:21:53.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:53 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-13T02:21:53.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:53 trial028 ceph-mon[26701]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 7.5 TiB / 7.5 TiB avail 2026-02-13T02:21:53.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:53 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:53.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:53 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:53.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:53 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:53.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:53 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:53.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:53 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:53.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:53 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:53.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:53 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:53.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:53 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:53.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:53 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:53.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:53 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2026-02-13T02:21:53.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:53 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:53.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:53 trial028 ceph-mon[26701]: osdmap e60: 12 total, 11 up, 12 in 2026-02-13T02:21:53.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:53 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-13T02:21:54.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:54 trial027 ceph-mon[24567]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 7.5 TiB / 7.5 TiB avail 2026-02-13T02:21:54.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:54 trial027 ceph-mon[24567]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 7.5 TiB / 7.5 TiB avail 2026-02-13T02:21:54.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:54 trial027 ceph-mon[24567]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 7.5 TiB / 7.5 TiB avail 2026-02-13T02:21:54.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:54 trial027 ceph-mon[24567]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:54.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:54 trial028 ceph-mon[26701]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 7.5 TiB / 7.5 TiB avail 2026-02-13T02:21:54.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:54 trial028 ceph-mon[26701]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 7.5 TiB / 7.5 TiB avail 2026-02-13T02:21:54.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:54 trial028 ceph-mon[26701]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 7.5 TiB / 7.5 TiB avail 2026-02-13T02:21:54.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:54 trial028 ceph-mon[26701]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:54.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:54 trial107 ceph-mon[27422]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 7.5 TiB / 7.5 TiB avail 2026-02-13T02:21:54.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:54 trial107 ceph-mon[27422]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 7.5 TiB / 7.5 TiB avail 2026-02-13T02:21:54.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:54 trial107 ceph-mon[27422]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 7.5 TiB / 7.5 TiB avail 2026-02-13T02:21:54.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:54 trial107 ceph-mon[27422]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:55.632 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:55 trial107 ceph-mon[27422]: osdmap e61: 12 total, 11 up, 12 in 2026-02-13T02:21:55.633 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:55 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-13T02:21:55.633 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:55 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:55.633 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:55 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:55.633 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:55 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:55.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:55 trial027 ceph-mon[24567]: osdmap e61: 12 total, 11 up, 12 in 2026-02-13T02:21:55.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:55 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-13T02:21:55.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:55 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:55.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:55 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:55.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:55 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:55.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:55 trial028 ceph-mon[26701]: osdmap e61: 12 total, 11 up, 12 in 2026-02-13T02:21:55.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:55 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-13T02:21:55.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:55 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:55.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:55 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:55.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:55 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:56.351 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:56.354+0000 7fa4297fa6c0 1 -- 10.20.193.107:0/2551076897 <== mgr.14116 v2:10.20.193.27:6800/1837452956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa42c04ca20 con 0x7fa4000b5600 2026-02-13T02:21:56.352 INFO:teuthology.orchestra.run.trial107.stdout:Created osd(s) 11 on host 'trial107' 2026-02-13T02:21:56.354 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:56.356+0000 7fa4324a96c0 1 -- 10.20.193.107:0/2551076897 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa4000b5600 msgr2=0x7fa4000d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:56.354 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:56.356+0000 7fa4324a96c0 1 --2- 10.20.193.107:0/2551076897 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa4000b5600 0x7fa4000d5ab0 secure :-1 s=READY pgs=131 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa418004750 tx=0x7fa418009260 comp rx=0 tx=0).stop 2026-02-13T02:21:56.354 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:56.356+0000 7fa4324a96c0 1 -- 10.20.193.107:0/2551076897 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa42c110040 msgr2=0x7fa42c171380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:56.354 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:56.356+0000 7fa4324a96c0 1 --2- 10.20.193.107:0/2551076897 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa42c110040 0x7fa42c171380 secure :-1 s=READY pgs=36 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa41c005dc0 tx=0x7fa41c00bbb0 comp rx=0 tx=0).stop 2026-02-13T02:21:56.354 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:56.356+0000 7fa4324a96c0 1 -- 10.20.193.107:0/2551076897 shutdown_connections 2026-02-13T02:21:56.354 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:56.356+0000 7fa4324a96c0 1 --2- 10.20.193.107:0/2551076897 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa4000b5600 0x7fa4000d5ab0 unknown :-1 s=CLOSED pgs=131 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:56.354 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:56.356+0000 7fa4324a96c0 1 --2- 10.20.193.107:0/2551076897 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa42c1109f0 0x7fa42c0f8e40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:56.354 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:56.356+0000 7fa4324a96c0 1 --2- 10.20.193.107:0/2551076897 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa42c110040 0x7fa42c171380 unknown :-1 s=CLOSED pgs=36 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:56.354 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:56.356+0000 7fa4324a96c0 1 --2- 10.20.193.107:0/2551076897 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa42c10eec0 0x7fa42c0f85e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:56.354 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:56.356+0000 7fa4324a96c0 1 -- 10.20.193.107:0/2551076897 >> 10.20.193.107:0/2551076897 conn(0x7fa42c0ec5c0 msgr2=0x7fa42c0eebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:21:56.354 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:56.356+0000 7fa4324a96c0 1 -- 10.20.193.107:0/2551076897 shutdown_connections 2026-02-13T02:21:56.354 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:21:56.356+0000 7fa4324a96c0 1 -- 10.20.193.107:0/2551076897 wait complete. 2026-02-13T02:21:56.394 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:56 trial107 ceph-mon[27422]: pgmap v158: 1 pgs: 1 peering; 577 KiB data, 296 MiB used, 7.5 TiB / 7.5 TiB avail; 185 KiB/s, 0 objects/s recovering 2026-02-13T02:21:56.394 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:56 trial107 ceph-mon[27422]: osdmap e62: 12 total, 11 up, 12 in 2026-02-13T02:21:56.394 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:56 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-13T02:21:56.394 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:56 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:56.394 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:56 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:56.394 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:56 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:56.394 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:56 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:56.394 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:56 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:56.394 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:56 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:56.394 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:56 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:56.394 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:56 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:56.394 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:56 trial107 ceph-mon[27422]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2026-02-13T02:21:56.394 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:56 trial107 ceph-mon[27422]: from='osd.11 [v2:10.20.193.107:6824/1496433214,v1:10.20.193.107:6825/1496433214]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2026-02-13T02:21:56.394 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:56 trial107 ceph-mon[27422]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:56.394 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:56 trial107 ceph-mon[27422]: Cluster is now healthy 2026-02-13T02:21:56.394 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:56 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:56.394 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:56 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:56.395 DEBUG:teuthology.orchestra.run.trial107:osd.11> sudo journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.11.service 2026-02-13T02:21:56.397 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2026-02-13T02:21:56.397 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd stat -f json 2026-02-13T02:21:56.519 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:21:56.634 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:56 trial027 ceph-mon[24567]: pgmap v158: 1 pgs: 1 peering; 577 KiB data, 296 MiB used, 7.5 TiB / 7.5 TiB avail; 185 KiB/s, 0 objects/s recovering 2026-02-13T02:21:56.634 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:56 trial027 ceph-mon[24567]: osdmap e62: 12 total, 11 up, 12 in 2026-02-13T02:21:56.634 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:56 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-13T02:21:56.634 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:56 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:56.634 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:56 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:56.634 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:56 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:56.634 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:56 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:56.634 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:56 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:56.634 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:56 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:56.634 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:56 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:56.634 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:56 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:56.634 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:56 trial027 ceph-mon[24567]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2026-02-13T02:21:56.634 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:56 trial027 ceph-mon[24567]: from='osd.11 [v2:10.20.193.107:6824/1496433214,v1:10.20.193.107:6825/1496433214]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2026-02-13T02:21:56.634 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:56 trial027 ceph-mon[24567]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:56.634 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:56 trial027 ceph-mon[24567]: Cluster is now healthy 2026-02-13T02:21:56.634 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:56 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:56.634 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:56 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:56.635 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.633+0000 7f5ca2c6c6c0 1 -- 10.20.193.27:0/4210597954 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5c9c1167b0 msgr2=0x7f5c9c116c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:56.635 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.633+0000 7f5ca2c6c6c0 1 --2- 10.20.193.27:0/4210597954 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5c9c1167b0 0x7f5c9c116c20 secure :-1 s=READY pgs=180 gs=1 cs=0 l=1 c_cookie=18365973581609399901 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5c84009970 tx=0x7f5c8402ed20 comp rx=0 tx=0).stop 2026-02-13T02:21:56.635 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.633+0000 7f5ca2c6c6c0 1 -- 10.20.193.27:0/4210597954 shutdown_connections 2026-02-13T02:21:56.635 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.633+0000 7f5ca2c6c6c0 1 --2- 10.20.193.27:0/4210597954 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5c9c117160 0x7f5c9c177af0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:56.635 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.633+0000 7f5ca2c6c6c0 1 --2- 10.20.193.27:0/4210597954 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5c9c1167b0 0x7f5c9c116c20 unknown :-1 s=CLOSED pgs=180 gs=1 cs=0 l=1 c_cookie=18365973581609399901 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:56.635 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.633+0000 7f5ca2c6c6c0 1 --2- 10.20.193.27:0/4210597954 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5c9c115630 0x7f5c9c115a20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:56.635 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.633+0000 7f5ca2c6c6c0 1 -- 10.20.193.27:0/4210597954 >> 10.20.193.27:0/4210597954 conn(0x7f5c9c05db80 msgr2=0x7f5c9c05df80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:21:56.635 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.633+0000 7f5ca2c6c6c0 1 -- 10.20.193.27:0/4210597954 shutdown_connections 2026-02-13T02:21:56.635 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.633+0000 7f5ca2c6c6c0 1 -- 10.20.193.27:0/4210597954 wait complete. 2026-02-13T02:21:56.635 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.633+0000 7f5ca2c6c6c0 1 Processor -- start 2026-02-13T02:21:56.635 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.633+0000 7f5ca2c6c6c0 1 -- start start 2026-02-13T02:21:56.635 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.633+0000 7f5ca2c6c6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5c9c115630 0x7f5c9c1002c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:56.635 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.633+0000 7f5ca2c6c6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5c9c1167b0 0x7f5c9c177af0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:56.635 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.633+0000 7f5ca2c6c6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5c9c117160 0x7f5c9c100b20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:56.635 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.633+0000 7f5ca2c6c6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c101750 con 0x7f5c9c115630 2026-02-13T02:21:56.635 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.633+0000 7f5ca2c6c6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c1018c0 con 0x7f5c9c1167b0 2026-02-13T02:21:56.635 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.633+0000 7f5ca2c6c6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c101a80 con 0x7f5c9c117160 2026-02-13T02:21:56.635 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.634+0000 7f5ca09e16c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5c9c115630 0x7f5c9c1002c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:56.635 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.634+0000 7f5ca09e16c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5c9c115630 0x7f5c9c1002c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:51978/0 (socket says 10.20.193.27:51978) 2026-02-13T02:21:56.635 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.634+0000 7f5ca09e16c0 1 -- 10.20.193.27:0/1285599314 learned_addr learned my addr 10.20.193.27:0/1285599314 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:21:56.636 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.634+0000 7f5c93fff6c0 1 --2- 10.20.193.27:0/1285599314 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5c9c1167b0 0x7f5c9c177af0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:56.636 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.634+0000 7f5ca11e26c0 1 --2- 10.20.193.27:0/1285599314 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5c9c117160 0x7f5c9c100b20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:56.636 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.634+0000 7f5ca09e16c0 1 -- 10.20.193.27:0/1285599314 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5c9c117160 msgr2=0x7f5c9c100b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:56.636 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.634+0000 7f5ca09e16c0 1 --2- 10.20.193.27:0/1285599314 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5c9c117160 0x7f5c9c100b20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:56.636 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.634+0000 7f5ca09e16c0 1 -- 10.20.193.27:0/1285599314 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5c9c1167b0 msgr2=0x7f5c9c177af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:56.636 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.634+0000 7f5ca09e16c0 1 --2- 10.20.193.27:0/1285599314 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5c9c1167b0 0x7f5c9c177af0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:56.636 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.634+0000 7f5ca09e16c0 1 -- 10.20.193.27:0/1285599314 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c84009590 con 0x7f5c9c115630 2026-02-13T02:21:56.636 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.634+0000 7f5ca09e16c0 1 --2- 10.20.193.27:0/1285599314 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5c9c115630 0x7f5c9c1002c0 secure :-1 s=READY pgs=181 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5c8c00b710 tx=0x7f5c8c00bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:21:56.636 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.634+0000 7f5c91ffb6c0 1 -- 10.20.193.27:0/1285599314 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5c8c004250 con 0x7f5c9c115630 2026-02-13T02:21:56.636 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.634+0000 7f5ca2c6c6c0 1 -- 10.20.193.27:0/1285599314 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c9c101d60 con 0x7f5c9c115630 2026-02-13T02:21:56.636 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.634+0000 7f5c91ffb6c0 1 -- 10.20.193.27:0/1285599314 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5c8c002d80 con 0x7f5c9c115630 2026-02-13T02:21:56.636 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.634+0000 7f5c91ffb6c0 1 -- 10.20.193.27:0/1285599314 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5c8c00cdc0 con 0x7f5c9c115630 2026-02-13T02:21:56.636 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.634+0000 7f5ca2c6c6c0 1 -- 10.20.193.27:0/1285599314 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c9c102240 con 0x7f5c9c115630 2026-02-13T02:21:56.636 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.634+0000 7f5ca2c6c6c0 1 -- 10.20.193.27:0/1285599314 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c9c17ba30 con 0x7f5c9c115630 2026-02-13T02:21:56.637 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.636+0000 7f5c91ffb6c0 1 -- 10.20.193.27:0/1285599314 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7f5c8c0028c0 con 0x7f5c9c115630 2026-02-13T02:21:56.638 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.637+0000 7f5c91ffb6c0 1 --2- 10.20.193.27:0/1285599314 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f5c780b5600 0x7f5c780d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:56.638 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.637+0000 7f5c93fff6c0 1 --2- 10.20.193.27:0/1285599314 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f5c780b5600 0x7f5c780d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:56.638 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.637+0000 7f5c91ffb6c0 1 -- 10.20.193.27:0/1285599314 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6586+0+0 (secure 0 0 0) 0x7f5c8c0e3580 con 0x7f5c9c115630 2026-02-13T02:21:56.638 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.637+0000 7f5c93fff6c0 1 --2- 10.20.193.27:0/1285599314 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f5c780b5600 0x7f5c780d5ab0 secure :-1 s=READY pgs=136 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5c8402fa20 tx=0x7f5c84005c60 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:21:56.638 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.637+0000 7f5c91ffb6c0 1 -- 10.20.193.27:0/1285599314 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f5c8c0a65a0 con 0x7f5c9c115630 2026-02-13T02:21:56.730 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.729+0000 7f5ca2c6c6c0 1 -- 10.20.193.27:0/1285599314 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f5c9c17b0d0 con 0x7f5c9c115630 2026-02-13T02:21:56.730 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.729+0000 7f5c91ffb6c0 1 -- 10.20.193.27:0/1285599314 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v62) ==== 74+0+133 (secure 0 0 0) 0x7f5c8c0a9750 con 0x7f5c9c115630 2026-02-13T02:21:56.730 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:21:56.731 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.730+0000 7f5ca2c6c6c0 1 -- 10.20.193.27:0/1285599314 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f5c780b5600 msgr2=0x7f5c780d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:56.731 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.730+0000 7f5ca2c6c6c0 1 --2- 10.20.193.27:0/1285599314 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f5c780b5600 0x7f5c780d5ab0 secure :-1 s=READY pgs=136 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5c8402fa20 tx=0x7f5c84005c60 comp rx=0 tx=0).stop 2026-02-13T02:21:56.731 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.730+0000 7f5ca2c6c6c0 1 -- 10.20.193.27:0/1285599314 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5c9c115630 msgr2=0x7f5c9c1002c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:56.731 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.730+0000 7f5ca2c6c6c0 1 --2- 10.20.193.27:0/1285599314 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5c9c115630 0x7f5c9c1002c0 secure :-1 s=READY pgs=181 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5c8c00b710 tx=0x7f5c8c00bbe0 comp rx=0 tx=0).stop 2026-02-13T02:21:56.731 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.730+0000 7f5ca2c6c6c0 1 -- 10.20.193.27:0/1285599314 shutdown_connections 2026-02-13T02:21:56.731 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.730+0000 7f5ca2c6c6c0 1 --2- 10.20.193.27:0/1285599314 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f5c780b5600 0x7f5c780d5ab0 unknown :-1 s=CLOSED pgs=136 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:56.731 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.730+0000 7f5ca2c6c6c0 1 --2- 10.20.193.27:0/1285599314 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5c9c117160 0x7f5c9c100b20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:56.731 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.730+0000 7f5ca2c6c6c0 1 --2- 10.20.193.27:0/1285599314 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5c9c1167b0 0x7f5c9c177af0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:56.731 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.730+0000 7f5ca2c6c6c0 1 --2- 10.20.193.27:0/1285599314 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5c9c115630 0x7f5c9c1002c0 unknown :-1 s=CLOSED pgs=181 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:56.731 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.730+0000 7f5ca2c6c6c0 1 -- 10.20.193.27:0/1285599314 >> 10.20.193.27:0/1285599314 conn(0x7f5c9c05db80 msgr2=0x7f5c9c0f6870 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:21:56.731 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.730+0000 7f5ca2c6c6c0 1 -- 10.20.193.27:0/1285599314 shutdown_connections 2026-02-13T02:21:56.731 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:56.730+0000 7f5ca2c6c6c0 1 -- 10.20.193.27:0/1285599314 wait complete. 2026-02-13T02:21:56.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:56 trial028 ceph-mon[26701]: pgmap v158: 1 pgs: 1 peering; 577 KiB data, 296 MiB used, 7.5 TiB / 7.5 TiB avail; 185 KiB/s, 0 objects/s recovering 2026-02-13T02:21:56.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:56 trial028 ceph-mon[26701]: osdmap e62: 12 total, 11 up, 12 in 2026-02-13T02:21:56.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:56 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-13T02:21:56.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:56 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:56.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:56 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:56.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:56 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:56.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:56 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:56.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:56 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:56.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:56 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:56.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:56 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:56.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:56 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:56.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:56 trial028 ceph-mon[26701]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2026-02-13T02:21:56.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:56 trial028 ceph-mon[26701]: from='osd.11 [v2:10.20.193.107:6824/1496433214,v1:10.20.193.107:6825/1496433214]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2026-02-13T02:21:56.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:56 trial028 ceph-mon[26701]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:21:56.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:56 trial028 ceph-mon[26701]: Cluster is now healthy 2026-02-13T02:21:56.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:56 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:56.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:56 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:56.775 INFO:teuthology.orchestra.run.trial027.stdout:{"epoch":62,"num_osds":12,"num_up_osds":11,"osd_up_since":1770949311,"num_in_osds":12,"osd_in_since":1770949310,"num_remapped_pgs":0} 2026-02-13T02:21:57.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:57 trial027 ceph-mon[24567]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 296 MiB used, 7.5 TiB / 7.5 TiB avail; 175 KiB/s, 0 objects/s recovering 2026-02-13T02:21:57.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:57.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:57 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1285599314' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2026-02-13T02:21:57.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:57 trial027 ceph-mon[24567]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2026-02-13T02:21:57.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:57 trial027 ceph-mon[24567]: osdmap e63: 12 total, 11 up, 12 in 2026-02-13T02:21:57.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-13T02:21:57.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:57 trial027 ceph-mon[24567]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.6836, "args": ["host=trial107", "root=default"]} : dispatch 2026-02-13T02:21:57.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:57 trial027 ceph-mon[24567]: from='osd.11 [v2:10.20.193.107:6824/1496433214,v1:10.20.193.107:6825/1496433214]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.6836, "args": ["host=trial107", "root=default"]} : dispatch 2026-02-13T02:21:57.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:57 trial028 ceph-mon[26701]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 296 MiB used, 7.5 TiB / 7.5 TiB avail; 175 KiB/s, 0 objects/s recovering 2026-02-13T02:21:57.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:57 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:57.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:57 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1285599314' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2026-02-13T02:21:57.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:57 trial028 ceph-mon[26701]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2026-02-13T02:21:57.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:57 trial028 ceph-mon[26701]: osdmap e63: 12 total, 11 up, 12 in 2026-02-13T02:21:57.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:57 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-13T02:21:57.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:57 trial028 ceph-mon[26701]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.6836, "args": ["host=trial107", "root=default"]} : dispatch 2026-02-13T02:21:57.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:57 trial028 ceph-mon[26701]: from='osd.11 [v2:10.20.193.107:6824/1496433214,v1:10.20.193.107:6825/1496433214]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.6836, "args": ["host=trial107", "root=default"]} : dispatch 2026-02-13T02:21:57.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:57 trial107 ceph-mon[27422]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 296 MiB used, 7.5 TiB / 7.5 TiB avail; 175 KiB/s, 0 objects/s recovering 2026-02-13T02:21:57.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:57 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:21:57.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:57 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1285599314' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2026-02-13T02:21:57.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:57 trial107 ceph-mon[27422]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2026-02-13T02:21:57.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:57 trial107 ceph-mon[27422]: osdmap e63: 12 total, 11 up, 12 in 2026-02-13T02:21:57.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:57 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-13T02:21:57.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:57 trial107 ceph-mon[27422]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.6836, "args": ["host=trial107", "root=default"]} : dispatch 2026-02-13T02:21:57.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:57 trial107 ceph-mon[27422]: from='osd.11 [v2:10.20.193.107:6824/1496433214,v1:10.20.193.107:6825/1496433214]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.6836, "args": ["host=trial107", "root=default"]} : dispatch 2026-02-13T02:21:57.776 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd stat -f json 2026-02-13T02:21:57.896 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:21:58.017 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.013+0000 7f0813dcd6c0 1 -- 10.20.193.27:0/2380339473 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f080c110a00 msgr2=0x7f080c171390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:58.017 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.013+0000 7f0813dcd6c0 1 --2- 10.20.193.27:0/2380339473 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f080c110a00 0x7f080c171390 secure :-1 s=READY pgs=182 gs=1 cs=0 l=1 c_cookie=10032870868067545817 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f080800aa40 tx=0x7f080802fde0 comp rx=0 tx=0).stop 2026-02-13T02:21:58.018 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.013+0000 7f0813dcd6c0 1 -- 10.20.193.27:0/2380339473 shutdown_connections 2026-02-13T02:21:58.018 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.013+0000 7f0813dcd6c0 1 --2- 10.20.193.27:0/2380339473 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f080c110a00 0x7f080c171390 unknown :-1 s=CLOSED pgs=182 gs=1 cs=0 l=1 c_cookie=10032870868067545817 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:58.018 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.013+0000 7f0813dcd6c0 1 --2- 10.20.193.27:0/2380339473 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f080c110050 0x7f080c1104c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:58.018 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.013+0000 7f0813dcd6c0 1 --2- 10.20.193.27:0/2380339473 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f080c10eed0 0x7f080c10f2c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:58.018 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.013+0000 7f0813dcd6c0 1 -- 10.20.193.27:0/2380339473 >> 10.20.193.27:0/2380339473 conn(0x7f080c0ec5f0 msgr2=0x7f080c10ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:21:58.018 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.013+0000 7f0813dcd6c0 1 -- 10.20.193.27:0/2380339473 shutdown_connections 2026-02-13T02:21:58.018 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.013+0000 7f0813dcd6c0 1 -- 10.20.193.27:0/2380339473 wait complete. 2026-02-13T02:21:58.018 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.014+0000 7f0813dcd6c0 1 Processor -- start 2026-02-13T02:21:58.019 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.014+0000 7f0813dcd6c0 1 -- start start 2026-02-13T02:21:58.019 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.014+0000 7f0813dcd6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f080c10eed0 0x7f080c0f8560 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:58.019 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.014+0000 7f0813dcd6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f080c110050 0x7f080c171390 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:58.019 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.014+0000 7f0813dcd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f080c110a00 0x7f080c0f8dc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:58.019 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.014+0000 7f0813dcd6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f080c0f99f0 con 0x7f080c110a00 2026-02-13T02:21:58.019 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.014+0000 7f0813dcd6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f080c0f9b60 con 0x7f080c10eed0 2026-02-13T02:21:58.019 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.014+0000 7f0813dcd6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f080c0f9d20 con 0x7f080c110050 2026-02-13T02:21:58.019 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.014+0000 7f081233f6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f080c110a00 0x7f080c0f8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:58.019 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.014+0000 7f081233f6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f080c110a00 0x7f080c0f8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:54084/0 (socket says 10.20.193.27:54084) 2026-02-13T02:21:58.019 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.014+0000 7f081233f6c0 1 -- 10.20.193.27:0/349835027 learned_addr learned my addr 10.20.193.27:0/349835027 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:21:58.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.014+0000 7f0811b3e6c0 1 --2- 10.20.193.27:0/349835027 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f080c10eed0 0x7f080c0f8560 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:58.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.014+0000 7f081133d6c0 1 --2- 10.20.193.27:0/349835027 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f080c110050 0x7f080c171390 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:58.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.014+0000 7f081233f6c0 1 -- 10.20.193.27:0/349835027 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f080c110050 msgr2=0x7f080c171390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:58.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.014+0000 7f081233f6c0 1 --2- 10.20.193.27:0/349835027 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f080c110050 0x7f080c171390 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:58.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.014+0000 7f081233f6c0 1 -- 10.20.193.27:0/349835027 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f080c10eed0 msgr2=0x7f080c0f8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:58.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.014+0000 7f081233f6c0 1 --2- 10.20.193.27:0/349835027 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f080c10eed0 0x7f080c0f8560 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:58.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.014+0000 7f081233f6c0 1 -- 10.20.193.27:0/349835027 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080800a660 con 0x7f080c110a00 2026-02-13T02:21:58.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.014+0000 7f081233f6c0 1 --2- 10.20.193.27:0/349835027 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f080c110a00 0x7f080c0f8dc0 secure :-1 s=READY pgs=183 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f080800a390 tx=0x7f0808005530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:21:58.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.014+0000 7f07faffd6c0 1 -- 10.20.193.27:0/349835027 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f08080057e0 con 0x7f080c110a00 2026-02-13T02:21:58.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.014+0000 7f0813dcd6c0 1 -- 10.20.193.27:0/349835027 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f080c0f9fa0 con 0x7f080c110a00 2026-02-13T02:21:58.021 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.014+0000 7f07faffd6c0 1 -- 10.20.193.27:0/349835027 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f0808003c40 con 0x7f080c110a00 2026-02-13T02:21:58.021 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.014+0000 7f07faffd6c0 1 -- 10.20.193.27:0/349835027 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0808038380 con 0x7f080c110a00 2026-02-13T02:21:58.021 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.014+0000 7f0813dcd6c0 1 -- 10.20.193.27:0/349835027 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f080c0fa420 con 0x7f080c110a00 2026-02-13T02:21:58.021 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.015+0000 7f0813dcd6c0 1 -- 10.20.193.27:0/349835027 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f080c175250 con 0x7f080c110a00 2026-02-13T02:21:58.022 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.016+0000 7f07faffd6c0 1 -- 10.20.193.27:0/349835027 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7f0808008260 con 0x7f080c110a00 2026-02-13T02:21:58.022 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.017+0000 7f07faffd6c0 1 --2- 10.20.193.27:0/349835027 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f07e80b5600 0x7f07e80d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:58.022 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.017+0000 7f0811b3e6c0 1 --2- 10.20.193.27:0/349835027 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f07e80b5600 0x7f07e80d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:58.022 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.017+0000 7f07faffd6c0 1 -- 10.20.193.27:0/349835027 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6624+0+0 (secure 0 0 0) 0x7f0808108830 con 0x7f080c110a00 2026-02-13T02:21:58.022 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.017+0000 7f07faffd6c0 1 -- 10.20.193.27:0/349835027 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f08080ce820 con 0x7f080c110a00 2026-02-13T02:21:58.023 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.017+0000 7f0811b3e6c0 1 --2- 10.20.193.27:0/349835027 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f07e80b5600 0x7f07e80d5ab0 secure :-1 s=READY pgs=138 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f07fc004750 tx=0x7f07fc009260 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:21:58.073 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:21:57 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:21:57.968+0000 7fc6a234c6c0 -1 osd.11 0 waiting for initial osdmap 2026-02-13T02:21:58.073 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:21:57 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:21:57.975+0000 7fc69e2106c0 -1 osd.11 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-13T02:21:58.109 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.107+0000 7f0813dcd6c0 1 -- 10.20.193.27:0/349835027 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f080c1748f0 con 0x7f080c110a00 2026-02-13T02:21:58.109 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.107+0000 7f07faffd6c0 1 -- 10.20.193.27:0/349835027 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v64) ==== 74+0+133 (secure 0 0 0) 0x7f08080cea00 con 0x7f080c110a00 2026-02-13T02:21:58.110 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:21:58.111 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.108+0000 7f0813dcd6c0 1 -- 10.20.193.27:0/349835027 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f07e80b5600 msgr2=0x7f07e80d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:58.111 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.108+0000 7f0813dcd6c0 1 --2- 10.20.193.27:0/349835027 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f07e80b5600 0x7f07e80d5ab0 secure :-1 s=READY pgs=138 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f07fc004750 tx=0x7f07fc009260 comp rx=0 tx=0).stop 2026-02-13T02:21:58.111 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.108+0000 7f0813dcd6c0 1 -- 10.20.193.27:0/349835027 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f080c110a00 msgr2=0x7f080c0f8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:58.111 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.108+0000 7f0813dcd6c0 1 --2- 10.20.193.27:0/349835027 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f080c110a00 0x7f080c0f8dc0 secure :-1 s=READY pgs=183 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f080800a390 tx=0x7f0808005530 comp rx=0 tx=0).stop 2026-02-13T02:21:58.111 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.108+0000 7f0813dcd6c0 1 -- 10.20.193.27:0/349835027 shutdown_connections 2026-02-13T02:21:58.112 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.108+0000 7f0813dcd6c0 1 --2- 10.20.193.27:0/349835027 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f07e80b5600 0x7f07e80d5ab0 unknown :-1 s=CLOSED pgs=138 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:58.112 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.108+0000 7f0813dcd6c0 1 --2- 10.20.193.27:0/349835027 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f080c110a00 0x7f080c0f8dc0 unknown :-1 s=CLOSED pgs=183 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:58.112 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.108+0000 7f0813dcd6c0 1 --2- 10.20.193.27:0/349835027 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f080c110050 0x7f080c171390 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:58.112 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.108+0000 7f0813dcd6c0 1 --2- 10.20.193.27:0/349835027 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f080c10eed0 0x7f080c0f8560 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:58.112 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.108+0000 7f0813dcd6c0 1 -- 10.20.193.27:0/349835027 >> 10.20.193.27:0/349835027 conn(0x7f080c0ec5f0 msgr2=0x7f080c0eeb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:21:58.112 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.108+0000 7f0813dcd6c0 1 -- 10.20.193.27:0/349835027 shutdown_connections 2026-02-13T02:21:58.112 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:58.108+0000 7f0813dcd6c0 1 -- 10.20.193.27:0/349835027 wait complete. 2026-02-13T02:21:58.151 INFO:teuthology.orchestra.run.trial027.stdout:{"epoch":64,"num_osds":12,"num_up_osds":11,"osd_up_since":1770949311,"num_in_osds":12,"osd_in_since":1770949310,"num_remapped_pgs":0} 2026-02-13T02:21:58.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:58 trial027 ceph-mon[24567]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 296 MiB used, 7.5 TiB / 7.5 TiB avail; 167 KiB/s, 0 objects/s recovering 2026-02-13T02:21:58.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:58 trial027 ceph-mon[24567]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:58.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:58.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:58.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:58.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:58.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:58.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:58.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:58.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:58.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:58.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:58.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:58 trial027 ceph-mon[24567]: pgmap v163: 1 pgs: 1 peering; 577 KiB data, 296 MiB used, 7.5 TiB / 7.5 TiB avail; 184 KiB/s, 0 objects/s recovering 2026-02-13T02:21:58.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:58.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:58.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:58.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:58 trial027 ceph-mon[24567]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.6836, "args": ["host=trial107", "root=default"]}]': finished 2026-02-13T02:21:58.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:58 trial027 ceph-mon[24567]: osdmap e64: 12 total, 11 up, 12 in 2026-02-13T02:21:58.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-13T02:21:58.661 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-13T02:21:58.661 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:58 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/349835027' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2026-02-13T02:21:58.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:58 trial028 ceph-mon[26701]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 296 MiB used, 7.5 TiB / 7.5 TiB avail; 167 KiB/s, 0 objects/s recovering 2026-02-13T02:21:58.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:58 trial028 ceph-mon[26701]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:58.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:58.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:58.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:58.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:58.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:58.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:58.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:58.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:58.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:58.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:58.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:58 trial028 ceph-mon[26701]: pgmap v163: 1 pgs: 1 peering; 577 KiB data, 296 MiB used, 7.5 TiB / 7.5 TiB avail; 184 KiB/s, 0 objects/s recovering 2026-02-13T02:21:58.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:58.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:58.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:58.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:58 trial028 ceph-mon[26701]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.6836, "args": ["host=trial107", "root=default"]}]': finished 2026-02-13T02:21:58.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:58 trial028 ceph-mon[26701]: osdmap e64: 12 total, 11 up, 12 in 2026-02-13T02:21:58.746 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-13T02:21:58.746 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-13T02:21:58.746 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:58 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/349835027' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2026-02-13T02:21:58.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:58 trial107 ceph-mon[27422]: pgmap v161: 1 pgs: 1 peering; 577 KiB data, 296 MiB used, 7.5 TiB / 7.5 TiB avail; 167 KiB/s, 0 objects/s recovering 2026-02-13T02:21:58.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:58 trial107 ceph-mon[27422]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:21:58.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:58.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:58.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:58.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:58.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:58.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2026-02-13T02:21:58.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:58.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:58.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:21:58.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:21:58.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:58 trial107 ceph-mon[27422]: pgmap v163: 1 pgs: 1 peering; 577 KiB data, 296 MiB used, 7.5 TiB / 7.5 TiB avail; 184 KiB/s, 0 objects/s recovering 2026-02-13T02:21:58.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:21:58.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:21:58.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:21:58.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:58 trial107 ceph-mon[27422]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.6836, "args": ["host=trial107", "root=default"]}]': finished 2026-02-13T02:21:58.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:58 trial107 ceph-mon[27422]: osdmap e64: 12 total, 11 up, 12 in 2026-02-13T02:21:58.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-13T02:21:58.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-13T02:21:58.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:58 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/349835027' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2026-02-13T02:21:59.153 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd stat -f json 2026-02-13T02:21:59.269 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:21:59.393 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.388+0000 7f0c06c6b6c0 1 -- 10.20.193.27:0/378102214 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0c001109f0 msgr2=0x7f0c00171380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:59.393 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.388+0000 7f0c06c6b6c0 1 --2- 10.20.193.27:0/378102214 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0c001109f0 0x7f0c00171380 secure :-1 s=READY pgs=184 gs=1 cs=0 l=1 c_cookie=13519914811935821710 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0bfc00aa40 tx=0x7f0bfc02fde0 comp rx=0 tx=0).stop 2026-02-13T02:21:59.393 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.388+0000 7f0c06c6b6c0 1 -- 10.20.193.27:0/378102214 shutdown_connections 2026-02-13T02:21:59.393 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.388+0000 7f0c06c6b6c0 1 --2- 10.20.193.27:0/378102214 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0c001109f0 0x7f0c00171380 unknown :-1 s=CLOSED pgs=184 gs=1 cs=0 l=1 c_cookie=13519914811935821710 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:59.393 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.388+0000 7f0c06c6b6c0 1 --2- 10.20.193.27:0/378102214 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0c00110040 0x7f0c001104b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:59.393 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.388+0000 7f0c06c6b6c0 1 --2- 10.20.193.27:0/378102214 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f0c0010eec0 0x7f0c0010f2b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:59.393 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.388+0000 7f0c06c6b6c0 1 -- 10.20.193.27:0/378102214 >> 10.20.193.27:0/378102214 conn(0x7f0c000ec5c0 msgr2=0x7f0c0010c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:21:59.394 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.388+0000 7f0c06c6b6c0 1 -- 10.20.193.27:0/378102214 shutdown_connections 2026-02-13T02:21:59.394 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.388+0000 7f0c06c6b6c0 1 -- 10.20.193.27:0/378102214 wait complete. 2026-02-13T02:21:59.394 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.389+0000 7f0c06c6b6c0 1 Processor -- start 2026-02-13T02:21:59.394 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.389+0000 7f0c06c6b6c0 1 -- start start 2026-02-13T02:21:59.394 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.389+0000 7f0c06c6b6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0c0010eec0 0x7f0c000f85e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:59.394 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.389+0000 7f0c06c6b6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f0c00110040 0x7f0c00171380 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:59.394 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.389+0000 7f0c06c6b6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0c001109f0 0x7f0c000f8e40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:59.394 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.389+0000 7f0c06c6b6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0c000f9a70 con 0x7f0c0010eec0 2026-02-13T02:21:59.394 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.389+0000 7f0c06c6b6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0c000f9be0 con 0x7f0c001109f0 2026-02-13T02:21:59.395 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.389+0000 7f0c06c6b6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0c000f9da0 con 0x7f0c00110040 2026-02-13T02:21:59.395 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.389+0000 7f0c049dc6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0c0010eec0 0x7f0c000f85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:59.395 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.389+0000 7f0c049dc6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0c0010eec0 0x7f0c000f85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:54104/0 (socket says 10.20.193.27:54104) 2026-02-13T02:21:59.395 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.389+0000 7f0c049dc6c0 1 -- 10.20.193.27:0/3647486966 learned_addr learned my addr 10.20.193.27:0/3647486966 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:21:59.395 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.389+0000 7f0bf7fff6c0 1 --2- 10.20.193.27:0/3647486966 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f0c00110040 0x7f0c00171380 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:59.395 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.389+0000 7f0c049dc6c0 1 -- 10.20.193.27:0/3647486966 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f0c00110040 msgr2=0x7f0c00171380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:59.395 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.389+0000 7f0c049dc6c0 1 --2- 10.20.193.27:0/3647486966 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f0c00110040 0x7f0c00171380 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:59.395 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.389+0000 7f0c049dc6c0 1 -- 10.20.193.27:0/3647486966 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0c001109f0 msgr2=0x7f0c000f8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:59.395 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.389+0000 7f0c049dc6c0 1 --2- 10.20.193.27:0/3647486966 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0c001109f0 0x7f0c000f8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:59.395 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.389+0000 7f0c049dc6c0 1 -- 10.20.193.27:0/3647486966 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bfc00a660 con 0x7f0c0010eec0 2026-02-13T02:21:59.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.389+0000 7f0c049dc6c0 1 --2- 10.20.193.27:0/3647486966 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0c0010eec0 0x7f0c000f85e0 secure :-1 s=READY pgs=185 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0bf000b710 tx=0x7f0bf000bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:21:59.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.389+0000 7f0bf5ffb6c0 1 -- 10.20.193.27:0/3647486966 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0bf0004250 con 0x7f0c0010eec0 2026-02-13T02:21:59.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.389+0000 7f0bf5ffb6c0 1 -- 10.20.193.27:0/3647486966 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f0bf0002d80 con 0x7f0c0010eec0 2026-02-13T02:21:59.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.389+0000 7f0c06c6b6c0 1 -- 10.20.193.27:0/3647486966 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c000fa080 con 0x7f0c0010eec0 2026-02-13T02:21:59.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.389+0000 7f0bf5ffb6c0 1 -- 10.20.193.27:0/3647486966 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0bf000cdc0 con 0x7f0c0010eec0 2026-02-13T02:21:59.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.389+0000 7f0c06c6b6c0 1 -- 10.20.193.27:0/3647486966 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c000fa5b0 con 0x7f0c0010eec0 2026-02-13T02:21:59.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.390+0000 7f0c06c6b6c0 1 -- 10.20.193.27:0/3647486966 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c00175240 con 0x7f0c0010eec0 2026-02-13T02:21:59.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.391+0000 7f0bf5ffb6c0 1 -- 10.20.193.27:0/3647486966 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7f0bf00028c0 con 0x7f0c0010eec0 2026-02-13T02:21:59.397 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.392+0000 7f0bf5ffb6c0 1 --2- 10.20.193.27:0/3647486966 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f0bdc0b5680 0x7f0bdc0d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:59.397 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.392+0000 7f0bf7fff6c0 1 --2- 10.20.193.27:0/3647486966 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f0bdc0b5680 0x7f0bdc0d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:59.397 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.392+0000 7f0bf5ffb6c0 1 -- 10.20.193.27:0/3647486966 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6904+0+0 (secure 0 0 0) 0x7f0bf00e3660 con 0x7f0c0010eec0 2026-02-13T02:21:59.397 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.392+0000 7f0bf7fff6c0 1 --2- 10.20.193.27:0/3647486966 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f0bdc0b5680 0x7f0bdc0d5b30 secure :-1 s=READY pgs=140 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0be8005fd0 tx=0x7f0be8005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:21:59.398 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.392+0000 7f0bf5ffb6c0 1 -- 10.20.193.27:0/3647486966 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f0bf00a6580 con 0x7f0c0010eec0 2026-02-13T02:21:59.487 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.486+0000 7f0c06c6b6c0 1 -- 10.20.193.27:0/3647486966 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f0c0010f2b0 con 0x7f0c0010eec0 2026-02-13T02:21:59.488 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.486+0000 7f0bf5ffb6c0 1 -- 10.20.193.27:0/3647486966 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v65) ==== 74+0+133 (secure 0 0 0) 0x7f0bf00a9730 con 0x7f0c0010eec0 2026-02-13T02:21:59.488 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:21:59.489 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.487+0000 7f0c06c6b6c0 1 -- 10.20.193.27:0/3647486966 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f0bdc0b5680 msgr2=0x7f0bdc0d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:59.489 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.487+0000 7f0c06c6b6c0 1 --2- 10.20.193.27:0/3647486966 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f0bdc0b5680 0x7f0bdc0d5b30 secure :-1 s=READY pgs=140 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0be8005fd0 tx=0x7f0be8005910 comp rx=0 tx=0).stop 2026-02-13T02:21:59.489 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.487+0000 7f0c06c6b6c0 1 -- 10.20.193.27:0/3647486966 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0c0010eec0 msgr2=0x7f0c000f85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:59.489 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.487+0000 7f0c06c6b6c0 1 --2- 10.20.193.27:0/3647486966 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0c0010eec0 0x7f0c000f85e0 secure :-1 s=READY pgs=185 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0bf000b710 tx=0x7f0bf000bbe0 comp rx=0 tx=0).stop 2026-02-13T02:21:59.489 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.487+0000 7f0c06c6b6c0 1 -- 10.20.193.27:0/3647486966 shutdown_connections 2026-02-13T02:21:59.490 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.487+0000 7f0c06c6b6c0 1 --2- 10.20.193.27:0/3647486966 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f0bdc0b5680 0x7f0bdc0d5b30 unknown :-1 s=CLOSED pgs=140 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:59.490 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.487+0000 7f0c06c6b6c0 1 --2- 10.20.193.27:0/3647486966 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0c001109f0 0x7f0c000f8e40 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:59.490 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.487+0000 7f0c06c6b6c0 1 --2- 10.20.193.27:0/3647486966 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f0c00110040 0x7f0c00171380 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:59.490 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.487+0000 7f0c06c6b6c0 1 --2- 10.20.193.27:0/3647486966 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0c0010eec0 0x7f0c000f85e0 unknown :-1 s=CLOSED pgs=185 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:59.490 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.487+0000 7f0c06c6b6c0 1 -- 10.20.193.27:0/3647486966 >> 10.20.193.27:0/3647486966 conn(0x7f0c000ec5c0 msgr2=0x7f0c000eebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:21:59.490 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.487+0000 7f0c06c6b6c0 1 -- 10.20.193.27:0/3647486966 shutdown_connections 2026-02-13T02:21:59.490 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.487+0000 7f0c06c6b6c0 1 -- 10.20.193.27:0/3647486966 wait complete. 2026-02-13T02:21:59.530 INFO:teuthology.orchestra.run.trial027.stdout:{"epoch":65,"num_osds":12,"num_up_osds":12,"osd_up_since":1770949318,"num_in_osds":12,"osd_in_since":1770949310,"num_remapped_pgs":0} 2026-02-13T02:21:59.531 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd dump --format=json 2026-02-13T02:21:59.648 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:21:59.763 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:59 trial027 ceph-mon[24567]: purged_snaps scrub starts 2026-02-13T02:21:59.763 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:59 trial027 ceph-mon[24567]: purged_snaps scrub ok 2026-02-13T02:21:59.763 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:59 trial027 ceph-mon[24567]: pgmap v165: 1 pgs: 1 peering; 577 KiB data, 296 MiB used, 7.5 TiB / 7.5 TiB avail 2026-02-13T02:21:59.763 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:59 trial027 ceph-mon[24567]: osd.11 [v2:10.20.193.107:6824/1496433214,v1:10.20.193.107:6825/1496433214] boot 2026-02-13T02:21:59.763 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:59 trial027 ceph-mon[24567]: osdmap e65: 12 total, 12 up, 12 in 2026-02-13T02:21:59.764 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:59 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-13T02:21:59.764 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:21:59 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3647486966' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2026-02-13T02:21:59.772 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.768+0000 7fc7252c06c0 1 -- 10.20.193.27:0/2262589565 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc720115440 msgr2=0x7fc720175de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:59.773 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.768+0000 7fc7252c06c0 1 --2- 10.20.193.27:0/2262589565 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc720115440 0x7fc720175de0 secure :-1 s=READY pgs=186 gs=1 cs=0 l=1 c_cookie=17732412839630446741 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc71000aa70 tx=0x7fc71002fde0 comp rx=0 tx=0).stop 2026-02-13T02:21:59.773 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.768+0000 7fc7252c06c0 1 -- 10.20.193.27:0/2262589565 shutdown_connections 2026-02-13T02:21:59.773 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.768+0000 7fc7252c06c0 1 --2- 10.20.193.27:0/2262589565 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc720115440 0x7fc720175de0 unknown :-1 s=CLOSED pgs=186 gs=1 cs=0 l=1 c_cookie=17732412839630446741 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:59.773 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.768+0000 7fc7252c06c0 1 --2- 10.20.193.27:0/2262589565 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc72010fc20 0x7fc720114c60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:59.773 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.768+0000 7fc7252c06c0 1 --2- 10.20.193.27:0/2262589565 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc72010f230 0x7fc72010f600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:59.773 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.768+0000 7fc7252c06c0 1 -- 10.20.193.27:0/2262589565 >> 10.20.193.27:0/2262589565 conn(0x7fc7200ecb30 msgr2=0x7fc72010cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:21:59.773 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.768+0000 7fc7252c06c0 1 -- 10.20.193.27:0/2262589565 shutdown_connections 2026-02-13T02:21:59.774 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.768+0000 7fc7252c06c0 1 -- 10.20.193.27:0/2262589565 wait complete. 2026-02-13T02:21:59.774 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.769+0000 7fc7252c06c0 1 Processor -- start 2026-02-13T02:21:59.774 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.769+0000 7fc7252c06c0 1 -- start start 2026-02-13T02:21:59.774 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.769+0000 7fc7252c06c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc72010f230 0x7fc72011d1f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:59.774 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.769+0000 7fc7252c06c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc720115440 0x7fc72011d730 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:59.774 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.769+0000 7fc7252c06c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc720122af0 0x7fc720122f90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:59.774 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.769+0000 7fc7252c06c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc720123500 con 0x7fc72010f230 2026-02-13T02:21:59.774 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.769+0000 7fc7252c06c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc720123670 con 0x7fc720115440 2026-02-13T02:21:59.774 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.769+0000 7fc7252c06c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc720123830 con 0x7fc720122af0 2026-02-13T02:21:59.774 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.769+0000 7fc71effd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc72010f230 0x7fc72011d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:59.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.769+0000 7fc71effd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc72010f230 0x7fc72011d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:54120/0 (socket says 10.20.193.27:54120) 2026-02-13T02:21:59.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.769+0000 7fc71effd6c0 1 -- 10.20.193.27:0/2029987376 learned_addr learned my addr 10.20.193.27:0/2029987376 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:21:59.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.769+0000 7fc71f7fe6c0 1 --2- 10.20.193.27:0/2029987376 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc720122af0 0x7fc720122f90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:59.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.769+0000 7fc71e7fc6c0 1 --2- 10.20.193.27:0/2029987376 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc720115440 0x7fc72011d730 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:59.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.769+0000 7fc71effd6c0 1 -- 10.20.193.27:0/2029987376 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc720122af0 msgr2=0x7fc720122f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:59.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.769+0000 7fc71effd6c0 1 --2- 10.20.193.27:0/2029987376 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc720122af0 0x7fc720122f90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:59.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.769+0000 7fc71effd6c0 1 -- 10.20.193.27:0/2029987376 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc720115440 msgr2=0x7fc72011d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:59.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.769+0000 7fc71effd6c0 1 --2- 10.20.193.27:0/2029987376 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc720115440 0x7fc72011d730 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:59.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.769+0000 7fc71effd6c0 1 -- 10.20.193.27:0/2029987376 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71000a660 con 0x7fc72010f230 2026-02-13T02:21:59.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.769+0000 7fc71effd6c0 1 --2- 10.20.193.27:0/2029987376 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc72010f230 0x7fc72011d1f0 secure :-1 s=READY pgs=187 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc71400b6c0 tx=0x7fc71400bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:21:59.776 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.769+0000 7fc6fffff6c0 1 -- 10.20.193.27:0/2029987376 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc7140044a0 con 0x7fc72010f230 2026-02-13T02:21:59.776 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.769+0000 7fc6fffff6c0 1 -- 10.20.193.27:0/2029987376 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc714009030 con 0x7fc72010f230 2026-02-13T02:21:59.776 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.769+0000 7fc6fffff6c0 1 -- 10.20.193.27:0/2029987376 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc71400cbe0 con 0x7fc72010f230 2026-02-13T02:21:59.776 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.769+0000 7fc7252c06c0 1 -- 10.20.193.27:0/2029987376 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc720163ce0 con 0x7fc72010f230 2026-02-13T02:21:59.776 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.769+0000 7fc7252c06c0 1 -- 10.20.193.27:0/2029987376 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc720164170 con 0x7fc72010f230 2026-02-13T02:21:59.776 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.770+0000 7fc7252c06c0 1 -- 10.20.193.27:0/2029987376 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc720110500 con 0x7fc72010f230 2026-02-13T02:21:59.776 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.771+0000 7fc6fffff6c0 1 -- 10.20.193.27:0/2029987376 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7fc714005080 con 0x7fc72010f230 2026-02-13T02:21:59.777 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.772+0000 7fc6fffff6c0 1 --2- 10.20.193.27:0/2029987376 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc6f40b5680 0x7fc6f40d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:21:59.777 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.772+0000 7fc71e7fc6c0 1 --2- 10.20.193.27:0/2029987376 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc6f40b5680 0x7fc6f40d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:21:59.778 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.772+0000 7fc6fffff6c0 1 -- 10.20.193.27:0/2029987376 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6904+0+0 (secure 0 0 0) 0x7fc7140e34e0 con 0x7fc72010f230 2026-02-13T02:21:59.778 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.772+0000 7fc6fffff6c0 1 -- 10.20.193.27:0/2029987376 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fc7140e39d0 con 0x7fc72010f230 2026-02-13T02:21:59.778 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.773+0000 7fc71e7fc6c0 1 --2- 10.20.193.27:0/2029987376 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc6f40b5680 0x7fc6f40d5b30 secure :-1 s=READY pgs=141 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc708005980 tx=0x7fc708005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:21:59.865 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.863+0000 7fc7252c06c0 1 -- 10.20.193.27:0/2029987376 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc72010fc20 con 0x7fc72010f230 2026-02-13T02:21:59.866 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.864+0000 7fc6fffff6c0 1 -- 10.20.193.27:0/2029987376 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+18118 (secure 0 0 0) 0x7fc7140a6400 con 0x7fc72010f230 2026-02-13T02:21:59.867 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:21:59.867 INFO:teuthology.orchestra.run.trial027.stdout:{"epoch":65,"fsid":"707dbb29-0882-11f1-9262-d404e6e7d460","created":"2026-02-13T02:19:50.420575+0000","modified":"2026-02-13T02:21:58.960121+0000","last_up_change":"2026-02-13T02:21:58.960121+0000","last_in_change":"2026-02-13T02:21:50.695981+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-13T02:20:54.702258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1a7af29f-2999-4be3-a38e-3fc7feb9f2f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":7,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6802","nonce":832083434},{"type":"v1","addr":"10.20.193.27:6803","nonce":832083434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6804","nonce":832083434},{"type":"v1","addr":"10.20.193.27:6805","nonce":832083434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6808","nonce":832083434},{"type":"v1","addr":"10.20.193.27:6809","nonce":832083434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6806","nonce":832083434},{"type":"v1","addr":"10.20.193.27:6807","nonce":832083434}]},"public_addr":"10.20.193.27:6803/832083434","cluster_addr":"10.20.193.27:6805/832083434","heartbeat_back_addr":"10.20.193.27:6809/832083434","heartbeat_front_addr":"10.20.193.27:6807/832083434","state":["exists","up"]},{"osd":1,"uuid":"8c0ad8cf-4150-43f6-8a9c-258b0ebf6a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6800","nonce":4070516162},{"type":"v1","addr":"10.20.193.28:6801","nonce":4070516162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6802","nonce":4070516162},{"type":"v1","addr":"10.20.193.28:6803","nonce":4070516162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6806","nonce":4070516162},{"type":"v1","addr":"10.20.193.28:6807","nonce":4070516162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6804","nonce":4070516162},{"type":"v1","addr":"10.20.193.28:6805","nonce":4070516162}]},"public_addr":"10.20.193.28:6801/4070516162","cluster_addr":"10.20.193.28:6803/4070516162","heartbeat_back_addr":"10.20.193.28:6807/4070516162","heartbeat_front_addr":"10.20.193.28:6805/4070516162","state":["exists","up"]},{"osd":2,"uuid":"a7813078-94d6-4559-a89a-5525161ab5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6800","nonce":4075625103},{"type":"v1","addr":"10.20.193.107:6801","nonce":4075625103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6802","nonce":4075625103},{"type":"v1","addr":"10.20.193.107:6803","nonce":4075625103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6806","nonce":4075625103},{"type":"v1","addr":"10.20.193.107:6807","nonce":4075625103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6804","nonce":4075625103},{"type":"v1","addr":"10.20.193.107:6805","nonce":4075625103}]},"public_addr":"10.20.193.107:6801/4075625103","cluster_addr":"10.20.193.107:6803/4075625103","heartbeat_back_addr":"10.20.193.107:6807/4075625103","heartbeat_front_addr":"10.20.193.107:6805/4075625103","state":["exists","up"]},{"osd":3,"uuid":"268f9714-faed-452b-ae34-3240fa23153a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6810","nonce":382071678},{"type":"v1","addr":"10.20.193.27:6811","nonce":382071678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6812","nonce":382071678},{"type":"v1","addr":"10.20.193.27:6813","nonce":382071678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6816","nonce":382071678},{"type":"v1","addr":"10.20.193.27:6817","nonce":382071678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6814","nonce":382071678},{"type":"v1","addr":"10.20.193.27:6815","nonce":382071678}]},"public_addr":"10.20.193.27:6811/382071678","cluster_addr":"10.20.193.27:6813/382071678","heartbeat_back_addr":"10.20.193.27:6817/382071678","heartbeat_front_addr":"10.20.193.27:6815/382071678","state":["exists","up"]},{"osd":4,"uuid":"4beaec0d-7b39-4bb0-9770-3dc4c848b106","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6808","nonce":711804032},{"type":"v1","addr":"10.20.193.28:6809","nonce":711804032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6810","nonce":711804032},{"type":"v1","addr":"10.20.193.28:6811","nonce":711804032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6814","nonce":711804032},{"type":"v1","addr":"10.20.193.28:6815","nonce":711804032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6812","nonce":711804032},{"type":"v1","addr":"10.20.193.28:6813","nonce":711804032}]},"public_addr":"10.20.193.28:6809/711804032","cluster_addr":"10.20.193.28:6811/711804032","heartbeat_back_addr":"10.20.193.28:6815/711804032","heartbeat_front_addr":"10.20.193.28:6813/711804032","state":["exists","up"]},{"osd":5,"uuid":"bb16754e-66f2-4951-9f31-d1dde591ec92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6808","nonce":4246100128},{"type":"v1","addr":"10.20.193.107:6809","nonce":4246100128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6810","nonce":4246100128},{"type":"v1","addr":"10.20.193.107:6811","nonce":4246100128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6814","nonce":4246100128},{"type":"v1","addr":"10.20.193.107:6815","nonce":4246100128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6812","nonce":4246100128},{"type":"v1","addr":"10.20.193.107:6813","nonce":4246100128}]},"public_addr":"10.20.193.107:6809/4246100128","cluster_addr":"10.20.193.107:6811/4246100128","heartbeat_back_addr":"10.20.193.107:6815/4246100128","heartbeat_front_addr":"10.20.193.107:6813/4246100128","state":["exists","up"]},{"osd":6,"uuid":"318afff7-b7c9-4b19-a7ba-06e7ab122ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6818","nonce":3514139178},{"type":"v1","addr":"10.20.193.27:6819","nonce":3514139178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6820","nonce":3514139178},{"type":"v1","addr":"10.20.193.27:6821","nonce":3514139178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6824","nonce":3514139178},{"type":"v1","addr":"10.20.193.27:6825","nonce":3514139178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6822","nonce":3514139178},{"type":"v1","addr":"10.20.193.27:6823","nonce":3514139178}]},"public_addr":"10.20.193.27:6819/3514139178","cluster_addr":"10.20.193.27:6821/3514139178","heartbeat_back_addr":"10.20.193.27:6825/3514139178","heartbeat_front_addr":"10.20.193.27:6823/3514139178","state":["exists","up"]},{"osd":7,"uuid":"b545439a-3f32-4b0a-bc6a-17f0d8050c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6816","nonce":1405200379},{"type":"v1","addr":"10.20.193.28:6817","nonce":1405200379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6818","nonce":1405200379},{"type":"v1","addr":"10.20.193.28:6819","nonce":1405200379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6822","nonce":1405200379},{"type":"v1","addr":"10.20.193.28:6823","nonce":1405200379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6820","nonce":1405200379},{"type":"v1","addr":"10.20.193.28:6821","nonce":1405200379}]},"public_addr":"10.20.193.28:6817/1405200379","cluster_addr":"10.20.193.28:6819/1405200379","heartbeat_back_addr":"10.20.193.28:6823/1405200379","heartbeat_front_addr":"10.20.193.28:6821/1405200379","state":["exists","up"]},{"osd":8,"uuid":"a7c6bbba-760c-484a-8171-44c70f7d9c89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6816","nonce":1331562104},{"type":"v1","addr":"10.20.193.107:6817","nonce":1331562104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6818","nonce":1331562104},{"type":"v1","addr":"10.20.193.107:6819","nonce":1331562104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6822","nonce":1331562104},{"type":"v1","addr":"10.20.193.107:6823","nonce":1331562104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6820","nonce":1331562104},{"type":"v1","addr":"10.20.193.107:6821","nonce":1331562104}]},"public_addr":"10.20.193.107:6817/1331562104","cluster_addr":"10.20.193.107:6819/1331562104","heartbeat_back_addr":"10.20.193.107:6823/1331562104","heartbeat_front_addr":"10.20.193.107:6821/1331562104","state":["exists","up"]},{"osd":9,"uuid":"72cbd422-3c3a-4904-9075-f543671d4f48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6826","nonce":1444927853},{"type":"v1","addr":"10.20.193.27:6827","nonce":1444927853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6828","nonce":1444927853},{"type":"v1","addr":"10.20.193.27:6829","nonce":1444927853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6832","nonce":1444927853},{"type":"v1","addr":"10.20.193.27:6833","nonce":1444927853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6830","nonce":1444927853},{"type":"v1","addr":"10.20.193.27:6831","nonce":1444927853}]},"public_addr":"10.20.193.27:6827/1444927853","cluster_addr":"10.20.193.27:6829/1444927853","heartbeat_back_addr":"10.20.193.27:6833/1444927853","heartbeat_front_addr":"10.20.193.27:6831/1444927853","state":["exists","up"]},{"osd":10,"uuid":"4bb7b75e-adf7-4c38-9343-dba513b57ed7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6824","nonce":2255755797},{"type":"v1","addr":"10.20.193.28:6825","nonce":2255755797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6826","nonce":2255755797},{"type":"v1","addr":"10.20.193.28:6827","nonce":2255755797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6830","nonce":2255755797},{"type":"v1","addr":"10.20.193.28:6831","nonce":2255755797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6828","nonce":2255755797},{"type":"v1","addr":"10.20.193.28:6829","nonce":2255755797}]},"public_addr":"10.20.193.28:6825/2255755797","cluster_addr":"10.20.193.28:6827/2255755797","heartbeat_back_addr":"10.20.193.28:6831/2255755797","heartbeat_front_addr":"10.20.193.28:6829/2255755797","state":["exists","up"]},{"osd":11,"uuid":"115fe8c6-77fc-41dd-ade0-db8eea024974","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6824","nonce":1496433214},{"type":"v1","addr":"10.20.193.107:6825","nonce":1496433214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6826","nonce":1496433214},{"type":"v1","addr":"10.20.193.107:6827","nonce":1496433214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6830","nonce":1496433214},{"type":"v1","addr":"10.20.193.107:6831","nonce":1496433214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6828","nonce":1496433214},{"type":"v1","addr":"10.20.193.107:6829","nonce":1496433214}]},"public_addr":"10.20.193.107:6825/1496433214","cluster_addr":"10.20.193.107:6827/1496433214","heartbeat_back_addr":"10.20.193.107:6831/1496433214","heartbeat_front_addr":"10.20.193.107:6829/1496433214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:20:37.596432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:20:44.867765+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:20:51.765273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:20:58.994342+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:06.232104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:13.195531+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:20.453172+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:27.721147+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:35.051979+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:42.463998+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:49.889642+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"10.20.193.27:0/1486586326":"2026-02-14T02:19:56.309429+0000","10.20.193.27:6801/3576730225":"2026-02-14T02:19:56.309429+0000","10.20.193.27:6800/3576730225":"2026-02-14T02:19:56.309429+0000","10.20.193.27:0/2609227888":"2026-02-14T02:19:56.309429+0000","10.20.193.27:0/1389740324":"2026-02-14T02:19:56.309429+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-13T02:21:59.868 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.865+0000 7fc7252c06c0 1 -- 10.20.193.27:0/2029987376 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc6f40b5680 msgr2=0x7fc6f40d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:59.868 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.865+0000 7fc7252c06c0 1 --2- 10.20.193.27:0/2029987376 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc6f40b5680 0x7fc6f40d5b30 secure :-1 s=READY pgs=141 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc708005980 tx=0x7fc708005910 comp rx=0 tx=0).stop 2026-02-13T02:21:59.868 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.865+0000 7fc7252c06c0 1 -- 10.20.193.27:0/2029987376 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc72010f230 msgr2=0x7fc72011d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:21:59.868 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.865+0000 7fc7252c06c0 1 --2- 10.20.193.27:0/2029987376 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc72010f230 0x7fc72011d1f0 secure :-1 s=READY pgs=187 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc71400b6c0 tx=0x7fc71400bb90 comp rx=0 tx=0).stop 2026-02-13T02:21:59.869 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.865+0000 7fc7252c06c0 1 -- 10.20.193.27:0/2029987376 shutdown_connections 2026-02-13T02:21:59.869 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.865+0000 7fc7252c06c0 1 --2- 10.20.193.27:0/2029987376 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc6f40b5680 0x7fc6f40d5b30 unknown :-1 s=CLOSED pgs=141 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:59.869 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.865+0000 7fc7252c06c0 1 --2- 10.20.193.27:0/2029987376 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc720122af0 0x7fc720122f90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:59.869 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.865+0000 7fc7252c06c0 1 --2- 10.20.193.27:0/2029987376 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc720115440 0x7fc72011d730 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:59.869 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.865+0000 7fc7252c06c0 1 --2- 10.20.193.27:0/2029987376 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc72010f230 0x7fc72011d1f0 unknown :-1 s=CLOSED pgs=187 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:21:59.869 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.865+0000 7fc7252c06c0 1 -- 10.20.193.27:0/2029987376 >> 10.20.193.27:0/2029987376 conn(0x7fc7200ecb30 msgr2=0x7fc72010cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:21:59.869 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.865+0000 7fc7252c06c0 1 -- 10.20.193.27:0/2029987376 shutdown_connections 2026-02-13T02:21:59.869 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:21:59.865+0000 7fc7252c06c0 1 -- 10.20.193.27:0/2029987376 wait complete. 2026-02-13T02:21:59.908 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2026-02-13T02:20:54.702258+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'nonprimary_shards': '{}', 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2026-02-13T02:21:59.909 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd pool get .mgr pg_num 2026-02-13T02:21:59.993 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:59 trial028 ceph-mon[26701]: purged_snaps scrub starts 2026-02-13T02:21:59.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:59 trial028 ceph-mon[26701]: purged_snaps scrub ok 2026-02-13T02:21:59.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:59 trial028 ceph-mon[26701]: pgmap v165: 1 pgs: 1 peering; 577 KiB data, 296 MiB used, 7.5 TiB / 7.5 TiB avail 2026-02-13T02:21:59.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:59 trial028 ceph-mon[26701]: osd.11 [v2:10.20.193.107:6824/1496433214,v1:10.20.193.107:6825/1496433214] boot 2026-02-13T02:21:59.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:59 trial028 ceph-mon[26701]: osdmap e65: 12 total, 12 up, 12 in 2026-02-13T02:21:59.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:59 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-13T02:21:59.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:21:59 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3647486966' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2026-02-13T02:22:00.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:59 trial107 ceph-mon[27422]: purged_snaps scrub starts 2026-02-13T02:22:00.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:59 trial107 ceph-mon[27422]: purged_snaps scrub ok 2026-02-13T02:22:00.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:59 trial107 ceph-mon[27422]: pgmap v165: 1 pgs: 1 peering; 577 KiB data, 296 MiB used, 7.5 TiB / 7.5 TiB avail 2026-02-13T02:22:00.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:59 trial107 ceph-mon[27422]: osd.11 [v2:10.20.193.107:6824/1496433214,v1:10.20.193.107:6825/1496433214] boot 2026-02-13T02:22:00.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:59 trial107 ceph-mon[27422]: osdmap e65: 12 total, 12 up, 12 in 2026-02-13T02:22:00.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:59 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-13T02:22:00.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:21:59 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3647486966' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2026-02-13T02:22:00.025 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:00.150 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.146+0000 7f16177946c0 1 -- 10.20.193.27:0/565819263 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f161010fbf0 msgr2=0x7f1610114c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:00.150 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.146+0000 7f16177946c0 1 --2- 10.20.193.27:0/565819263 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f161010fbf0 0x7f1610114c30 secure :-1 s=READY pgs=188 gs=1 cs=0 l=1 c_cookie=11499353903016559483 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f16040099a0 tx=0x7f160402ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:00.150 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.146+0000 7f16177946c0 1 -- 10.20.193.27:0/565819263 shutdown_connections 2026-02-13T02:22:00.150 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.146+0000 7f16177946c0 1 --2- 10.20.193.27:0/565819263 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1610115410 0x7f1610175db0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:00.151 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.146+0000 7f16177946c0 1 --2- 10.20.193.27:0/565819263 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f161010fbf0 0x7f1610114c30 unknown :-1 s=CLOSED pgs=188 gs=1 cs=0 l=1 c_cookie=11499353903016559483 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:00.151 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.146+0000 7f16177946c0 1 --2- 10.20.193.27:0/565819263 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f161010f200 0x7f161010f5d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:00.151 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.146+0000 7f16177946c0 1 -- 10.20.193.27:0/565819263 >> 10.20.193.27:0/565819263 conn(0x7f16100ecb00 msgr2=0x7f161010cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:00.151 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.146+0000 7f16177946c0 1 -- 10.20.193.27:0/565819263 shutdown_connections 2026-02-13T02:22:00.151 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.146+0000 7f16177946c0 1 -- 10.20.193.27:0/565819263 wait complete. 2026-02-13T02:22:00.151 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.146+0000 7f16177946c0 1 Processor -- start 2026-02-13T02:22:00.151 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.146+0000 7f16177946c0 1 -- start start 2026-02-13T02:22:00.152 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.147+0000 7f16177946c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f161010f200 0x7f16100f4c80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:00.152 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.147+0000 7f16177946c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1610115410 0x7f1610175cb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:00.152 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.147+0000 7f16177946c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f16100f52b0 0x7f16100fa580 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:00.152 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.147+0000 7f16177946c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f16100f59a0 con 0x7f16100f52b0 2026-02-13T02:22:00.152 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.147+0000 7f16177946c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f16100faac0 con 0x7f161010f200 2026-02-13T02:22:00.152 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.147+0000 7f16177946c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f16100fac80 con 0x7f1610115410 2026-02-13T02:22:00.152 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.147+0000 7f1615d066c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f16100f52b0 0x7f16100fa580 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:00.153 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.147+0000 7f1615d066c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f16100f52b0 0x7f16100fa580 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:54152/0 (socket says 10.20.193.27:54152) 2026-02-13T02:22:00.153 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.147+0000 7f1615d066c0 1 -- 10.20.193.27:0/3699734959 learned_addr learned my addr 10.20.193.27:0/3699734959 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:00.153 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.147+0000 7f1614d046c0 1 --2- 10.20.193.27:0/3699734959 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1610115410 0x7f1610175cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:00.153 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.147+0000 7f16155056c0 1 --2- 10.20.193.27:0/3699734959 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f161010f200 0x7f16100f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:00.153 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.147+0000 7f1615d066c0 1 -- 10.20.193.27:0/3699734959 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1610115410 msgr2=0x7f1610175cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:00.153 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.147+0000 7f1615d066c0 1 --2- 10.20.193.27:0/3699734959 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1610115410 0x7f1610175cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:00.153 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.147+0000 7f1615d066c0 1 -- 10.20.193.27:0/3699734959 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f161010f200 msgr2=0x7f16100f4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:00.154 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.147+0000 7f1615d066c0 1 --2- 10.20.193.27:0/3699734959 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f161010f200 0x7f16100f4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:00.154 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.147+0000 7f1615d066c0 1 -- 10.20.193.27:0/3699734959 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1604009590 con 0x7f16100f52b0 2026-02-13T02:22:00.154 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.147+0000 7f1615d066c0 1 --2- 10.20.193.27:0/3699734959 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f16100f52b0 0x7f16100fa580 secure :-1 s=READY pgs=189 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f160c00c6c0 tx=0x7f160c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:00.154 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.147+0000 7f15fe7fc6c0 1 -- 10.20.193.27:0/3699734959 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f160c003a30 con 0x7f16100f52b0 2026-02-13T02:22:00.154 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.147+0000 7f15fe7fc6c0 1 -- 10.20.193.27:0/3699734959 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f160c0054e0 con 0x7f16100f52b0 2026-02-13T02:22:00.154 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.147+0000 7f16177946c0 1 -- 10.20.193.27:0/3699734959 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16100faf60 con 0x7f16100f52b0 2026-02-13T02:22:00.154 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.147+0000 7f15fe7fc6c0 1 -- 10.20.193.27:0/3699734959 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f160c00dd10 con 0x7f16100f52b0 2026-02-13T02:22:00.155 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.147+0000 7f16177946c0 1 -- 10.20.193.27:0/3699734959 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16100fb6f0 con 0x7f16100f52b0 2026-02-13T02:22:00.155 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.147+0000 7f16177946c0 1 -- 10.20.193.27:0/3699734959 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16100fb410 con 0x7f16100f52b0 2026-02-13T02:22:00.156 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.149+0000 7f15fe7fc6c0 1 -- 10.20.193.27:0/3699734959 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7f160c005080 con 0x7f16100f52b0 2026-02-13T02:22:00.157 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.150+0000 7f15fe7fc6c0 1 --2- 10.20.193.27:0/3699734959 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f15ec0b5600 0x7f15ec0d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:00.157 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.150+0000 7f16155056c0 1 --2- 10.20.193.27:0/3699734959 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f15ec0b5600 0x7f15ec0d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:00.157 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.150+0000 7f15fe7fc6c0 1 -- 10.20.193.27:0/3699734959 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6904+0+0 (secure 0 0 0) 0x7f160c0e4400 con 0x7f16100f52b0 2026-02-13T02:22:00.157 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.150+0000 7f15fe7fc6c0 1 -- 10.20.193.27:0/3699734959 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f160c0e4880 con 0x7f16100f52b0 2026-02-13T02:22:00.157 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.150+0000 7f16155056c0 1 --2- 10.20.193.27:0/3699734959 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f15ec0b5600 0x7f15ec0d5ab0 secure :-1 s=READY pgs=142 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1600005980 tx=0x7f1600005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:00.244 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.242+0000 7f16177946c0 1 -- 10.20.193.27:0/3699734959 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f16100f5dd0 con 0x7f16100f52b0 2026-02-13T02:22:00.244 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.242+0000 7f15fe7fc6c0 1 -- 10.20.193.27:0/3699734959 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) ==== 93+0+10 (secure 0 0 0) 0x7f160c0a72e0 con 0x7f16100f52b0 2026-02-13T02:22:00.244 INFO:teuthology.orchestra.run.trial027.stdout:pg_num: 1 2026-02-13T02:22:00.245 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.243+0000 7f16177946c0 1 -- 10.20.193.27:0/3699734959 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f15ec0b5600 msgr2=0x7f15ec0d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:00.245 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.243+0000 7f16177946c0 1 --2- 10.20.193.27:0/3699734959 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f15ec0b5600 0x7f15ec0d5ab0 secure :-1 s=READY pgs=142 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1600005980 tx=0x7f1600005910 comp rx=0 tx=0).stop 2026-02-13T02:22:00.246 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.243+0000 7f16177946c0 1 -- 10.20.193.27:0/3699734959 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f16100f52b0 msgr2=0x7f16100fa580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:00.246 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.243+0000 7f16177946c0 1 --2- 10.20.193.27:0/3699734959 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f16100f52b0 0x7f16100fa580 secure :-1 s=READY pgs=189 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f160c00c6c0 tx=0x7f160c00cb90 comp rx=0 tx=0).stop 2026-02-13T02:22:00.246 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.243+0000 7f16177946c0 1 -- 10.20.193.27:0/3699734959 shutdown_connections 2026-02-13T02:22:00.246 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.243+0000 7f16177946c0 1 --2- 10.20.193.27:0/3699734959 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f15ec0b5600 0x7f15ec0d5ab0 unknown :-1 s=CLOSED pgs=142 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:00.246 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.243+0000 7f16177946c0 1 --2- 10.20.193.27:0/3699734959 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f16100f52b0 0x7f16100fa580 unknown :-1 s=CLOSED pgs=189 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:00.246 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.243+0000 7f16177946c0 1 --2- 10.20.193.27:0/3699734959 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1610115410 0x7f1610175cb0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:00.246 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.243+0000 7f16177946c0 1 --2- 10.20.193.27:0/3699734959 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f161010f200 0x7f16100f4c80 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:00.246 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.243+0000 7f16177946c0 1 -- 10.20.193.27:0/3699734959 >> 10.20.193.27:0/3699734959 conn(0x7f16100ecb00 msgr2=0x7f161010d900 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:00.246 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.243+0000 7f16177946c0 1 -- 10.20.193.27:0/3699734959 shutdown_connections 2026-02-13T02:22:00.246 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:00.243+0000 7f16177946c0 1 -- 10.20.193.27:0/3699734959 wait complete. 2026-02-13T02:22:00.285 INFO:tasks.cephadm:Adding mds.a on trial027 2026-02-13T02:22:00.285 INFO:tasks.cephadm:Adding mds.d on trial027 2026-02-13T02:22:00.285 INFO:tasks.cephadm:Adding mds.g on trial027 2026-02-13T02:22:00.285 INFO:tasks.cephadm:Adding mds.j on trial027 2026-02-13T02:22:00.285 INFO:tasks.cephadm:Adding mds.b on trial028 2026-02-13T02:22:00.285 INFO:tasks.cephadm:Adding mds.e on trial028 2026-02-13T02:22:00.286 INFO:tasks.cephadm:Adding mds.h on trial028 2026-02-13T02:22:00.286 INFO:tasks.cephadm:Adding mds.k on trial028 2026-02-13T02:22:00.286 INFO:tasks.cephadm:Adding mds.c on trial107 2026-02-13T02:22:00.286 INFO:tasks.cephadm:Adding mds.f on trial107 2026-02-13T02:22:00.286 INFO:tasks.cephadm:Adding mds.i on trial107 2026-02-13T02:22:00.286 INFO:tasks.cephadm:Adding mds.l on trial107 2026-02-13T02:22:00.286 DEBUG:teuthology.orchestra.run.trial107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph orch apply mds all '12;trial027=a;trial027=d;trial027=g;trial027=j;trial028=b;trial028=e;trial028=h;trial028=k;trial107=c;trial107=f;trial107=i;trial107=l' 2026-02-13T02:22:00.407 INFO:teuthology.orchestra.run.trial107.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.c/config 2026-02-13T02:22:00.525 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.525+0000 7f5a81ba86c0 1 -- 10.20.193.107:0/3126029530 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5a7c110040 msgr2=0x7f5a7c1104b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:00.525 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.525+0000 7f5a81ba86c0 1 --2- 10.20.193.107:0/3126029530 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5a7c110040 0x7f5a7c1104b0 secure :-1 s=READY pgs=44 gs=1 cs=0 l=1 c_cookie=15603003802201288099 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5a6c009970 tx=0x7f5a6c02ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:00.525 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.525+0000 7f5a81ba86c0 1 -- 10.20.193.107:0/3126029530 shutdown_connections 2026-02-13T02:22:00.525 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.525+0000 7f5a81ba86c0 1 --2- 10.20.193.107:0/3126029530 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5a7c1109f0 0x7f5a7c171380 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:00.525 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.525+0000 7f5a81ba86c0 1 --2- 10.20.193.107:0/3126029530 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5a7c110040 0x7f5a7c1104b0 unknown :-1 s=CLOSED pgs=44 gs=1 cs=0 l=1 c_cookie=15603003802201288099 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:00.525 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.525+0000 7f5a81ba86c0 1 --2- 10.20.193.107:0/3126029530 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5a7c10eec0 0x7f5a7c10f2b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:00.526 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.525+0000 7f5a81ba86c0 1 -- 10.20.193.107:0/3126029530 >> 10.20.193.107:0/3126029530 conn(0x7f5a7c0ec5c0 msgr2=0x7f5a7c10c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:00.526 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.525+0000 7f5a81ba86c0 1 -- 10.20.193.107:0/3126029530 shutdown_connections 2026-02-13T02:22:00.526 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.525+0000 7f5a81ba86c0 1 -- 10.20.193.107:0/3126029530 wait complete. 2026-02-13T02:22:00.526 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.525+0000 7f5a81ba86c0 1 Processor -- start 2026-02-13T02:22:00.526 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.525+0000 7f5a81ba86c0 1 -- start start 2026-02-13T02:22:00.526 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.525+0000 7f5a81ba86c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5a7c10eec0 0x7f5a7c0f8740 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:00.526 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.525+0000 7f5a81ba86c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5a7c110040 0x7f5a7c171380 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:00.526 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.525+0000 7f5a81ba86c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5a7c1109f0 0x7f5a7c0f8fa0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:00.526 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.525+0000 7f5a81ba86c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7c0f9bd0 con 0x7f5a7c110040 2026-02-13T02:22:00.526 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.525+0000 7f5a81ba86c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7c0f9d40 con 0x7f5a7c10eec0 2026-02-13T02:22:00.526 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.525+0000 7f5a81ba86c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7c0f9f00 con 0x7f5a7c1109f0 2026-02-13T02:22:00.526 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.526+0000 7f5a7bfff6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5a7c1109f0 0x7f5a7c0f8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:00.527 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.526+0000 7f5a7bfff6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5a7c1109f0 0x7f5a7c0f8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.107:3300/0 says I am v2:10.20.193.107:59912/0 (socket says 10.20.193.107:59912) 2026-02-13T02:22:00.527 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.526+0000 7f5a7bfff6c0 1 -- 10.20.193.107:0/2447217449 learned_addr learned my addr 10.20.193.107:0/2447217449 (peer_addr_for_me v2:10.20.193.107:0/0) 2026-02-13T02:22:00.527 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.526+0000 7f5a7affd6c0 1 --2- 10.20.193.107:0/2447217449 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5a7c110040 0x7f5a7c171380 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:00.527 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.526+0000 7f5a7b7fe6c0 1 --2- 10.20.193.107:0/2447217449 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5a7c10eec0 0x7f5a7c0f8740 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:00.527 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.526+0000 7f5a7affd6c0 1 -- 10.20.193.107:0/2447217449 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5a7c1109f0 msgr2=0x7f5a7c0f8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:00.527 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.526+0000 7f5a7affd6c0 1 --2- 10.20.193.107:0/2447217449 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5a7c1109f0 0x7f5a7c0f8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:00.527 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.526+0000 7f5a7affd6c0 1 -- 10.20.193.107:0/2447217449 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5a7c10eec0 msgr2=0x7f5a7c0f8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:00.527 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.526+0000 7f5a7affd6c0 1 --2- 10.20.193.107:0/2447217449 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5a7c10eec0 0x7f5a7c0f8740 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:00.527 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.526+0000 7f5a7affd6c0 1 -- 10.20.193.107:0/2447217449 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6c009590 con 0x7f5a7c110040 2026-02-13T02:22:00.527 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.526+0000 7f5a7affd6c0 1 --2- 10.20.193.107:0/2447217449 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5a7c110040 0x7f5a7c171380 secure :-1 s=READY pgs=190 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5a6c02fe90 tx=0x7f5a6c0045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:00.527 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.526+0000 7f5a78ff96c0 1 -- 10.20.193.107:0/2447217449 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5a6c002ba0 con 0x7f5a7c110040 2026-02-13T02:22:00.527 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.526+0000 7f5a78ff96c0 1 -- 10.20.193.107:0/2447217449 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5a6c002d00 con 0x7f5a7c110040 2026-02-13T02:22:00.528 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.526+0000 7f5a81ba86c0 1 -- 10.20.193.107:0/2447217449 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a7c0fa180 con 0x7f5a7c110040 2026-02-13T02:22:00.528 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.526+0000 7f5a78ff96c0 1 -- 10.20.193.107:0/2447217449 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5a6c0362f0 con 0x7f5a7c110040 2026-02-13T02:22:00.528 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.526+0000 7f5a81ba86c0 1 -- 10.20.193.107:0/2447217449 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a7c0fa650 con 0x7f5a7c110040 2026-02-13T02:22:00.528 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.527+0000 7f5a81ba86c0 1 -- 10.20.193.107:0/2447217449 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a7c175290 con 0x7f5a7c110040 2026-02-13T02:22:00.529 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.528+0000 7f5a78ff96c0 1 -- 10.20.193.107:0/2447217449 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7f5a6c0078a0 con 0x7f5a7c110040 2026-02-13T02:22:00.529 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.528+0000 7f5a78ff96c0 1 --2- 10.20.193.107:0/2447217449 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f5a500b5680 0x7f5a500d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:00.529 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.529+0000 7f5a78ff96c0 1 -- 10.20.193.107:0/2447217449 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6904+0+0 (secure 0 0 0) 0x7f5a6c1076a0 con 0x7f5a7c110040 2026-02-13T02:22:00.529 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.529+0000 7f5a7b7fe6c0 1 --2- 10.20.193.107:0/2447217449 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f5a500b5680 0x7f5a500d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:00.529 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.529+0000 7f5a78ff96c0 1 -- 10.20.193.107:0/2447217449 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f5a6c0ca580 con 0x7f5a7c110040 2026-02-13T02:22:00.529 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.529+0000 7f5a7b7fe6c0 1 --2- 10.20.193.107:0/2447217449 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f5a500b5680 0x7f5a500d5b30 secure :-1 s=READY pgs=143 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5a68004750 tx=0x7f5a68009390 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:00.621 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.623+0000 7f5a81ba86c0 1 -- 10.20.193.107:0/2447217449 --> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;trial027=a;trial027=d;trial027=g;trial027=j;trial028=b;trial028=e;trial028=h;trial028=k;trial107=c;trial107=f;trial107=i;trial107=l", "target": ["mon-mgr", ""]}) -- 0x7f5a7c04ca20 con 0x7f5a500b5680 2026-02-13T02:22:00.622 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:00 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2029987376' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-13T02:22:00.622 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:00 trial107 ceph-mon[27422]: osdmap e66: 12 total, 12 up, 12 in 2026-02-13T02:22:00.622 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:00 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3699734959' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2026-02-13T02:22:00.625 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.627+0000 7f5a78ff96c0 1 -- 10.20.193.107:0/2447217449 <== mgr.14116 v2:10.20.193.27:6800/1837452956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f5a7c04ca20 con 0x7f5a500b5680 2026-02-13T02:22:00.626 INFO:teuthology.orchestra.run.trial107.stdout:Scheduled mds.all update... 2026-02-13T02:22:00.627 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.629+0000 7f5a81ba86c0 1 -- 10.20.193.107:0/2447217449 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f5a500b5680 msgr2=0x7f5a500d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:00.627 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.629+0000 7f5a81ba86c0 1 --2- 10.20.193.107:0/2447217449 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f5a500b5680 0x7f5a500d5b30 secure :-1 s=READY pgs=143 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5a68004750 tx=0x7f5a68009390 comp rx=0 tx=0).stop 2026-02-13T02:22:00.627 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.629+0000 7f5a81ba86c0 1 -- 10.20.193.107:0/2447217449 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5a7c110040 msgr2=0x7f5a7c171380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:00.627 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.629+0000 7f5a81ba86c0 1 --2- 10.20.193.107:0/2447217449 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5a7c110040 0x7f5a7c171380 secure :-1 s=READY pgs=190 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5a6c02fe90 tx=0x7f5a6c0045f0 comp rx=0 tx=0).stop 2026-02-13T02:22:00.627 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.629+0000 7f5a81ba86c0 1 -- 10.20.193.107:0/2447217449 shutdown_connections 2026-02-13T02:22:00.627 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.629+0000 7f5a81ba86c0 1 --2- 10.20.193.107:0/2447217449 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f5a500b5680 0x7f5a500d5b30 unknown :-1 s=CLOSED pgs=143 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:00.627 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.629+0000 7f5a81ba86c0 1 --2- 10.20.193.107:0/2447217449 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5a7c1109f0 0x7f5a7c0f8fa0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:00.628 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.629+0000 7f5a81ba86c0 1 --2- 10.20.193.107:0/2447217449 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5a7c110040 0x7f5a7c171380 unknown :-1 s=CLOSED pgs=190 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:00.628 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.629+0000 7f5a81ba86c0 1 --2- 10.20.193.107:0/2447217449 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5a7c10eec0 0x7f5a7c0f8740 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:00.628 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.629+0000 7f5a81ba86c0 1 -- 10.20.193.107:0/2447217449 >> 10.20.193.107:0/2447217449 conn(0x7f5a7c0ec5c0 msgr2=0x7f5a7c0eec50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:00.628 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.629+0000 7f5a81ba86c0 1 -- 10.20.193.107:0/2447217449 shutdown_connections 2026-02-13T02:22:00.628 INFO:teuthology.orchestra.run.trial107.stderr:2026-02-13T02:22:00.629+0000 7f5a81ba86c0 1 -- 10.20.193.107:0/2447217449 wait complete. 2026-02-13T02:22:00.669 DEBUG:teuthology.orchestra.run.trial027:mds.a> sudo journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.a.service 2026-02-13T02:22:00.671 DEBUG:teuthology.orchestra.run.trial027:mds.d> sudo journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.d.service 2026-02-13T02:22:00.715 DEBUG:teuthology.orchestra.run.trial027:mds.g> sudo journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.g.service 2026-02-13T02:22:00.759 DEBUG:teuthology.orchestra.run.trial027:mds.j> sudo journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.j.service 2026-02-13T02:22:00.779 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:00 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2029987376' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-13T02:22:00.779 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:00 trial027 ceph-mon[24567]: osdmap e66: 12 total, 12 up, 12 in 2026-02-13T02:22:00.779 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:00 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3699734959' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2026-02-13T02:22:00.780 DEBUG:teuthology.orchestra.run.trial028:mds.b> sudo journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.b.service 2026-02-13T02:22:00.782 DEBUG:teuthology.orchestra.run.trial028:mds.e> sudo journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.e.service 2026-02-13T02:22:00.803 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:00 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2029987376' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-13T02:22:00.804 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:00 trial028 ceph-mon[26701]: osdmap e66: 12 total, 12 up, 12 in 2026-02-13T02:22:00.804 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:00 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3699734959' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2026-02-13T02:22:00.805 DEBUG:teuthology.orchestra.run.trial028:mds.h> sudo journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.h.service 2026-02-13T02:22:00.847 DEBUG:teuthology.orchestra.run.trial028:mds.k> sudo journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.k.service 2026-02-13T02:22:00.891 DEBUG:teuthology.orchestra.run.trial107:mds.c> sudo journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.c.service 2026-02-13T02:22:00.894 DEBUG:teuthology.orchestra.run.trial107:mds.f> sudo journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.f.service 2026-02-13T02:22:00.937 DEBUG:teuthology.orchestra.run.trial107:mds.i> sudo journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.i.service 2026-02-13T02:22:00.979 DEBUG:teuthology.orchestra.run.trial107:mds.l> sudo journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.l.service 2026-02-13T02:22:01.023 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2026-02-13T02:22:01.023 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph fs dump --format=json 2026-02-13T02:22:01.144 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:01.264 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.262+0000 7fe4af81d6c0 1 -- 10.20.193.27:0/4067394436 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe4a810f200 msgr2=0x7fe4a810f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:01.264 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.262+0000 7fe4af81d6c0 1 --2- 10.20.193.27:0/4067394436 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe4a810f200 0x7fe4a810f5d0 secure :-1 s=READY pgs=191 gs=1 cs=0 l=1 c_cookie=731578682103659468 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe4980099a0 tx=0x7fe49802ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:01.264 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.262+0000 7fe4af81d6c0 1 -- 10.20.193.27:0/4067394436 shutdown_connections 2026-02-13T02:22:01.264 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.262+0000 7fe4af81d6c0 1 --2- 10.20.193.27:0/4067394436 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe4a8115410 0x7fe4a8175db0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:01.264 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.262+0000 7fe4af81d6c0 1 --2- 10.20.193.27:0/4067394436 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe4a810fbf0 0x7fe4a8114c30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:01.264 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.262+0000 7fe4af81d6c0 1 --2- 10.20.193.27:0/4067394436 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe4a810f200 0x7fe4a810f5d0 unknown :-1 s=CLOSED pgs=191 gs=1 cs=0 l=1 c_cookie=731578682103659468 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:01.264 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.262+0000 7fe4af81d6c0 1 -- 10.20.193.27:0/4067394436 >> 10.20.193.27:0/4067394436 conn(0x7fe4a80ecb00 msgr2=0x7fe4a810cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:01.264 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.262+0000 7fe4af81d6c0 1 -- 10.20.193.27:0/4067394436 shutdown_connections 2026-02-13T02:22:01.264 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.262+0000 7fe4af81d6c0 1 -- 10.20.193.27:0/4067394436 wait complete. 2026-02-13T02:22:01.265 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.263+0000 7fe4af81d6c0 1 Processor -- start 2026-02-13T02:22:01.265 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.263+0000 7fe4af81d6c0 1 -- start start 2026-02-13T02:22:01.265 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.263+0000 7fe4af81d6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe4a810f200 0x7fe4a811d250 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:01.265 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.263+0000 7fe4af81d6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe4a8115410 0x7fe4a811d790 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:01.266 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.263+0000 7fe4af81d6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe4a8122b50 0x7fe4a8122ff0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:01.266 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.263+0000 7fe4af81d6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a8123560 con 0x7fe4a8115410 2026-02-13T02:22:01.266 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.263+0000 7fe4af81d6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a81236d0 con 0x7fe4a8122b50 2026-02-13T02:22:01.266 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.263+0000 7fe4af81d6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a8123890 con 0x7fe4a810f200 2026-02-13T02:22:01.266 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.263+0000 7fe4acd8d6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe4a8115410 0x7fe4a811d790 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:01.266 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.263+0000 7fe4acd8d6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe4a8115410 0x7fe4a811d790 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:54180/0 (socket says 10.20.193.27:54180) 2026-02-13T02:22:01.266 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.263+0000 7fe4acd8d6c0 1 -- 10.20.193.27:0/2664601123 learned_addr learned my addr 10.20.193.27:0/2664601123 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:01.266 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.263+0000 7fe4ad58e6c0 1 --2- 10.20.193.27:0/2664601123 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe4a810f200 0x7fe4a811d250 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:01.266 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.263+0000 7fe4add8f6c0 1 --2- 10.20.193.27:0/2664601123 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe4a8122b50 0x7fe4a8122ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:01.266 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.263+0000 7fe4acd8d6c0 1 -- 10.20.193.27:0/2664601123 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe4a810f200 msgr2=0x7fe4a811d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:01.266 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.263+0000 7fe4acd8d6c0 1 --2- 10.20.193.27:0/2664601123 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe4a810f200 0x7fe4a811d250 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:01.266 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.263+0000 7fe4acd8d6c0 1 -- 10.20.193.27:0/2664601123 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe4a8122b50 msgr2=0x7fe4a8122ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:01.266 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.263+0000 7fe4acd8d6c0 1 --2- 10.20.193.27:0/2664601123 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe4a8122b50 0x7fe4a8122ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:01.266 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.263+0000 7fe4acd8d6c0 1 -- 10.20.193.27:0/2664601123 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe498009590 con 0x7fe4a8115410 2026-02-13T02:22:01.266 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.263+0000 7fe4acd8d6c0 1 --2- 10.20.193.27:0/2664601123 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe4a8115410 0x7fe4a811d790 secure :-1 s=READY pgs=192 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe49c005dc0 tx=0x7fe49c00bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:01.266 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.263+0000 7fe4967fc6c0 1 -- 10.20.193.27:0/2664601123 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe49c004250 con 0x7fe4a8115410 2026-02-13T02:22:01.266 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.263+0000 7fe4af81d6c0 1 -- 10.20.193.27:0/2664601123 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4a8163d40 con 0x7fe4a8115410 2026-02-13T02:22:01.266 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.263+0000 7fe4967fc6c0 1 -- 10.20.193.27:0/2664601123 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fe49c009070 con 0x7fe4a8115410 2026-02-13T02:22:01.266 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.263+0000 7fe4967fc6c0 1 -- 10.20.193.27:0/2664601123 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe49c00cd50 con 0x7fe4a8115410 2026-02-13T02:22:01.266 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.263+0000 7fe4af81d6c0 1 -- 10.20.193.27:0/2664601123 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4a81641d0 con 0x7fe4a8115410 2026-02-13T02:22:01.266 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.264+0000 7fe4af81d6c0 1 -- 10.20.193.27:0/2664601123 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4a81104d0 con 0x7fe4a8115410 2026-02-13T02:22:01.267 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.265+0000 7fe4967fc6c0 1 -- 10.20.193.27:0/2664601123 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7fe49c0028c0 con 0x7fe4a8115410 2026-02-13T02:22:01.267 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.266+0000 7fe4967fc6c0 1 --2- 10.20.193.27:0/2664601123 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe4840b5600 0x7fe4840d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:01.267 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.266+0000 7fe4ad58e6c0 1 --2- 10.20.193.27:0/2664601123 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe4840b5600 0x7fe4840d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:01.267 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.266+0000 7fe4967fc6c0 1 -- 10.20.193.27:0/2664601123 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6904+0+0 (secure 0 0 0) 0x7fe49c0e3680 con 0x7fe4a8115410 2026-02-13T02:22:01.267 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.266+0000 7fe4967fc6c0 1 -- 10.20.193.27:0/2664601123 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fe49c0a58d0 con 0x7fe4a8115410 2026-02-13T02:22:01.267 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.266+0000 7fe4ad58e6c0 1 --2- 10.20.193.27:0/2664601123 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe4840b5600 0x7fe4840d5ab0 secure :-1 s=READY pgs=144 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe498006010 tx=0x7fe498005de0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:01.380 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.378+0000 7fe4af81d6c0 1 -- 10.20.193.27:0/2664601123 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4a810f5d0 con 0x7fe4a8115410 2026-02-13T02:22:01.380 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.379+0000 7fe4967fc6c0 1 -- 10.20.193.27:0/2664601123 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fe49c0a9570 con 0x7fe4a8115410 2026-02-13T02:22:01.381 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:22:01.381 INFO:teuthology.orchestra.run.trial027.stdout:{"epoch":1,"btime":"2026-02-13T02:19:50:420456+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2026-02-13T02:22:01.382 INFO:teuthology.orchestra.run.trial027.stderr:dumped fsmap epoch 1 2026-02-13T02:22:01.383 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.380+0000 7fe4af81d6c0 1 -- 10.20.193.27:0/2664601123 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe4840b5600 msgr2=0x7fe4840d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:01.383 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.380+0000 7fe4af81d6c0 1 --2- 10.20.193.27:0/2664601123 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe4840b5600 0x7fe4840d5ab0 secure :-1 s=READY pgs=144 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe498006010 tx=0x7fe498005de0 comp rx=0 tx=0).stop 2026-02-13T02:22:01.383 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.380+0000 7fe4af81d6c0 1 -- 10.20.193.27:0/2664601123 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe4a8115410 msgr2=0x7fe4a811d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:01.383 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.380+0000 7fe4af81d6c0 1 --2- 10.20.193.27:0/2664601123 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe4a8115410 0x7fe4a811d790 secure :-1 s=READY pgs=192 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe49c005dc0 tx=0x7fe49c00bbb0 comp rx=0 tx=0).stop 2026-02-13T02:22:01.383 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.380+0000 7fe4af81d6c0 1 -- 10.20.193.27:0/2664601123 shutdown_connections 2026-02-13T02:22:01.383 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.380+0000 7fe4af81d6c0 1 --2- 10.20.193.27:0/2664601123 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe4840b5600 0x7fe4840d5ab0 unknown :-1 s=CLOSED pgs=144 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:01.383 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.380+0000 7fe4af81d6c0 1 --2- 10.20.193.27:0/2664601123 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe4a8122b50 0x7fe4a8122ff0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:01.383 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.380+0000 7fe4af81d6c0 1 --2- 10.20.193.27:0/2664601123 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe4a8115410 0x7fe4a811d790 unknown :-1 s=CLOSED pgs=192 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:01.383 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.380+0000 7fe4af81d6c0 1 --2- 10.20.193.27:0/2664601123 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe4a810f200 0x7fe4a811d250 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:01.383 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.380+0000 7fe4af81d6c0 1 -- 10.20.193.27:0/2664601123 >> 10.20.193.27:0/2664601123 conn(0x7fe4a80ecb00 msgr2=0x7fe4a810cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:01.384 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.380+0000 7fe4af81d6c0 1 -- 10.20.193.27:0/2664601123 shutdown_connections 2026-02-13T02:22:01.384 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:01.380+0000 7fe4af81d6c0 1 -- 10.20.193.27:0/2664601123 wait complete. 2026-02-13T02:22:01.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:01 trial027 ceph-mon[24567]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:01.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:01 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:01.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:01 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:22:01.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:01 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:01.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:01 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:22:01.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:01 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:01.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:01 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:22:01.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:01 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:22:01.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:01 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:01.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:01 trial027 ceph-mon[24567]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:22:01.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:01 trial027 ceph-mon[24567]: Cluster is now healthy 2026-02-13T02:22:01.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:01 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:01.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:01 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:01.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:01 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:01.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:01 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2664601123' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-13T02:22:01.750 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:01 trial107 ceph-mon[27422]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:01.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:01 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:01.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:01 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:22:01.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:01 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:01.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:01 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:22:01.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:01 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:01.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:01 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:22:01.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:01 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:22:01.751 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:01 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:01.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:01 trial107 ceph-mon[27422]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:22:01.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:01 trial107 ceph-mon[27422]: Cluster is now healthy 2026-02-13T02:22:01.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:01 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:01.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:01 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:01.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:01 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:01.752 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:01 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2664601123' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-13T02:22:01.993 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:01 trial028 ceph-mon[26701]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:01.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:01 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:01.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:01 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:22:01.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:01 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:01.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:01 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:22:01.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:01 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:01.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:01 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:22:01.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:01 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:22:01.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:01 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:01.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:01 trial028 ceph-mon[26701]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:22:01.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:01 trial028 ceph-mon[26701]: Cluster is now healthy 2026-02-13T02:22:01.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:01 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:01.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:01 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:01.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:01 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:01.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:01 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2664601123' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-13T02:22:02.024 INFO:journalctl@ceph.mds.l.trial107.stdout:Feb 13 02:22:01 trial107 podman[55305]: 2026-02-13 02:22:01.995970995 +0000 UTC m=+0.016890673 container create e894fc7f2c71e7e4f1fbe10f1714446734c1cd3575cc1c5bf0e597f4b5ccaac4 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-l, license=BSD-3-Clause, ceph=True, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.version=10, OSD_FLAVOR=default, org.opencontainers.image.licenses=BSD-3-Clause, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, version=10, CEPH_REF=wip-pdonnell-testing-20260210.171210, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.authors=Ceph Release Team , CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, summary=Rocky Linux Base image, io.buildah.version=1.33.7, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, vendor=Rocky Enterprise Software Foundation, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, name=rockylinux, org.opencontainers.image.title=rockylinux) 2026-02-13T02:22:02.501 INFO:journalctl@ceph.mds.l.trial107.stdout:Feb 13 02:22:02 trial107 podman[55305]: 2026-02-13 02:22:02.029150014 +0000 UTC m=+0.050069712 container init e894fc7f2c71e7e4f1fbe10f1714446734c1cd3575cc1c5bf0e597f4b5ccaac4 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-l, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.licenses=BSD-3-Clause, org.opencontainers.image.title=rockylinux, license=BSD-3-Clause, ceph=True, CEPH_REF=wip-pdonnell-testing-20260210.171210, org.opencontainers.image.authors=Ceph Release Team , GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.version=10, summary=Rocky Linux Base image, OSD_FLAVOR=default, version=10, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, name=rockylinux, io.buildah.version=1.33.7, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, vendor=Rocky Enterprise Software Foundation, FROM_IMAGE=docker.io/rockylinux/rockylinux:10) 2026-02-13T02:22:02.501 INFO:journalctl@ceph.mds.l.trial107.stdout:Feb 13 02:22:02 trial107 podman[55305]: 2026-02-13 02:22:02.031793998 +0000 UTC m=+0.052713686 container start e894fc7f2c71e7e4f1fbe10f1714446734c1cd3575cc1c5bf0e597f4b5ccaac4 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-l, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, org.opencontainers.image.licenses=BSD-3-Clause, license=BSD-3-Clause, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.version=10, summary=Rocky Linux Base image, vendor=Rocky Enterprise Software Foundation, name=rockylinux, io.buildah.version=1.33.7, version=10, CEPH_REF=wip-pdonnell-testing-20260210.171210, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, org.opencontainers.image.title=rockylinux, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.authors=Ceph Release Team , CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, OSD_FLAVOR=default, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, ceph=True) 2026-02-13T02:22:02.501 INFO:journalctl@ceph.mds.l.trial107.stdout:Feb 13 02:22:02 trial107 bash[55305]: e894fc7f2c71e7e4f1fbe10f1714446734c1cd3575cc1c5bf0e597f4b5ccaac4 2026-02-13T02:22:02.501 INFO:journalctl@ceph.mds.l.trial107.stdout:Feb 13 02:22:02 trial107 podman[55305]: 2026-02-13 02:22:01.989096682 +0000 UTC m=+0.010016380 image pull b86a4a15c2dcd78db08124ee64cb48fcd681d6db032e313da19d063813b235da quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9 2026-02-13T02:22:02.501 INFO:journalctl@ceph.mds.l.trial107.stdout:Feb 13 02:22:02 trial107 systemd[1]: Started Ceph mds.l for 707dbb29-0882-11f1-9262-d404e6e7d460. 2026-02-13T02:22:02.501 INFO:journalctl@ceph.mds.l.trial107.stdout:Feb 13 02:22:02 trial107 ceph-mds[55329]: -- 10.20.193.107:0/1374075422 <== mon.1 v2:10.20.193.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dfb09ad860 con 0x55dfb0a63000 2026-02-13T02:22:02.501 INFO:journalctl@ceph.mds.l.trial107.stdout:Feb 13 02:22:02 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-l[55325]: starting mds.l at 2026-02-13T02:22:02.737 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:02 trial028 ceph-mon[26701]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;trial027=a;trial027=d;trial027=g;trial027=j;trial028=b;trial028=e;trial028=h;trial028=k;trial107=c;trial107=f;trial107=i;trial107=l", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:22:02.737 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:02 trial028 ceph-mon[26701]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:02.737 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:02 trial028 ceph-mon[26701]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:02.737 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:02 trial028 ceph-mon[26701]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:22:02.737 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:02 trial028 ceph-mon[26701]: mds.? [v2:10.20.193.107:6832/3112510413,v1:10.20.193.107:6833/3112510413] up:boot 2026-02-13T02:22:02.737 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:02 trial028 ceph-mon[26701]: fsmap 1 up:standby 2026-02-13T02:22:02.738 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:02 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2026-02-13T02:22:02.738 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:02 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:02.738 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:02 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:02.738 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:02 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:02.738 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:02 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:02.738 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:02 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:02.738 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:02 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:02.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:02 trial027 ceph-mon[24567]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;trial027=a;trial027=d;trial027=g;trial027=j;trial028=b;trial028=e;trial028=h;trial028=k;trial107=c;trial107=f;trial107=i;trial107=l", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:22:02.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:02 trial027 ceph-mon[24567]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:02.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:02 trial027 ceph-mon[24567]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:02.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:02 trial027 ceph-mon[24567]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:22:02.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:02 trial027 ceph-mon[24567]: mds.? [v2:10.20.193.107:6832/3112510413,v1:10.20.193.107:6833/3112510413] up:boot 2026-02-13T02:22:02.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:02 trial027 ceph-mon[24567]: fsmap 1 up:standby 2026-02-13T02:22:02.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:02 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2026-02-13T02:22:02.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:02 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:02.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:02 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:02.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:02 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:02.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:02 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:02.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:02 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:02.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:02 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:02.993 INFO:journalctl@ceph.mds.k.trial028.stdout:Feb 13 02:22:02 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-k[53896]: starting mds.k at 2026-02-13T02:22:03.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:02 trial107 ceph-mon[27422]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;trial027=a;trial027=d;trial027=g;trial027=j;trial028=b;trial028=e;trial028=h;trial028=k;trial107=c;trial107=f;trial107=i;trial107=l", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:22:03.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:02 trial107 ceph-mon[27422]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:03.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:02 trial107 ceph-mon[27422]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:03.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:02 trial107 ceph-mon[27422]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:22:03.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:02 trial107 ceph-mon[27422]: mds.? [v2:10.20.193.107:6832/3112510413,v1:10.20.193.107:6833/3112510413] up:boot 2026-02-13T02:22:03.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:02 trial107 ceph-mon[27422]: fsmap 1 up:standby 2026-02-13T02:22:03.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:02 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2026-02-13T02:22:03.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:02 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:03.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:02 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:03.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:02 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:03.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:02 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:03.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:02 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:03.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:02 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:03.428 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph fs dump --format=json 2026-02-13T02:22:03.652 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:03.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.771+0000 7feff4f456c0 1 -- 10.20.193.27:0/2497118351 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7feff0061050 msgr2=0x7feff0155180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:03.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.771+0000 7feff4f456c0 1 --2- 10.20.193.27:0/2497118351 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7feff0061050 0x7feff0155180 secure :-1 s=READY pgs=195 gs=1 cs=0 l=1 c_cookie=4000637854413748822 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fefd8009970 tx=0x7fefd802ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:03.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.772+0000 7feff4f456c0 1 -- 10.20.193.27:0/2497118351 shutdown_connections 2026-02-13T02:22:03.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.772+0000 7feff4f456c0 1 --2- 10.20.193.27:0/2497118351 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7feff01556c0 0x7feff0175aa0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:03.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.772+0000 7feff4f456c0 1 --2- 10.20.193.27:0/2497118351 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7feff0061050 0x7feff0155180 unknown :-1 s=CLOSED pgs=195 gs=1 cs=0 l=1 c_cookie=4000637854413748822 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:03.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.772+0000 7feff4f456c0 1 --2- 10.20.193.27:0/2497118351 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7feff0060740 0x7feff0060b10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:03.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.772+0000 7feff4f456c0 1 -- 10.20.193.27:0/2497118351 >> 10.20.193.27:0/2497118351 conn(0x7feff00ea8b0 msgr2=0x7feff010acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:03.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.772+0000 7feff4f456c0 1 -- 10.20.193.27:0/2497118351 shutdown_connections 2026-02-13T02:22:03.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.772+0000 7feff4f456c0 1 -- 10.20.193.27:0/2497118351 wait complete. 2026-02-13T02:22:03.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.772+0000 7feff4f456c0 1 Processor -- start 2026-02-13T02:22:03.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.772+0000 7feff4f456c0 1 -- start start 2026-02-13T02:22:03.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.772+0000 7feff4f456c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7feff0060740 0x7feff0161520 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:03.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.772+0000 7feff4f456c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7feff0061050 0x7feff0161a60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:03.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.772+0000 7feff4f456c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7feff01556c0 0x7feff0166e20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:03.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.772+0000 7feff4f456c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7feff0167470 con 0x7feff01556c0 2026-02-13T02:22:03.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.772+0000 7feff4f456c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7feff01675e0 con 0x7feff0060740 2026-02-13T02:22:03.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.772+0000 7feff4f456c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7feff01677a0 con 0x7feff0061050 2026-02-13T02:22:03.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.772+0000 7fefeed766c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7feff01556c0 0x7feff0166e20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:03.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.772+0000 7fefeed766c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7feff01556c0 0x7feff0166e20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:54216/0 (socket says 10.20.193.27:54216) 2026-02-13T02:22:03.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.772+0000 7fefeed766c0 1 -- 10.20.193.27:0/213075927 learned_addr learned my addr 10.20.193.27:0/213075927 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:03.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.772+0000 7fefedd746c0 1 --2- 10.20.193.27:0/213075927 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7feff0061050 0x7feff0161a60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:03.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.772+0000 7fefeed766c0 1 -- 10.20.193.27:0/213075927 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7feff0061050 msgr2=0x7feff0161a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:03.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.772+0000 7fefee5756c0 1 --2- 10.20.193.27:0/213075927 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7feff0060740 0x7feff0161520 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:03.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.772+0000 7fefeed766c0 1 --2- 10.20.193.27:0/213075927 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7feff0061050 0x7feff0161a60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:03.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.772+0000 7fefeed766c0 1 -- 10.20.193.27:0/213075927 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7feff0060740 msgr2=0x7feff0161520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:03.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.772+0000 7fefeed766c0 1 --2- 10.20.193.27:0/213075927 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7feff0060740 0x7feff0161520 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:03.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.772+0000 7fefeed766c0 1 -- 10.20.193.27:0/213075927 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd8009590 con 0x7feff01556c0 2026-02-13T02:22:03.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.772+0000 7fefee5756c0 1 --2- 10.20.193.27:0/213075927 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7feff0060740 0x7feff0161520 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:22:03.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.773+0000 7fefeed766c0 1 --2- 10.20.193.27:0/213075927 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7feff01556c0 0x7feff0166e20 secure :-1 s=READY pgs=196 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fefe000a2c0 tx=0x7fefe000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:03.776 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.773+0000 7fefd77fe6c0 1 -- 10.20.193.27:0/213075927 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fefe0005280 con 0x7feff01556c0 2026-02-13T02:22:03.776 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.773+0000 7fefd77fe6c0 1 -- 10.20.193.27:0/213075927 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fefe0003cd0 con 0x7feff01556c0 2026-02-13T02:22:03.776 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.773+0000 7feff4f456c0 1 -- 10.20.193.27:0/213075927 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff0167a80 con 0x7feff01556c0 2026-02-13T02:22:03.776 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.773+0000 7fefd77fe6c0 1 -- 10.20.193.27:0/213075927 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fefe000dc30 con 0x7feff01556c0 2026-02-13T02:22:03.776 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.773+0000 7feff4f456c0 1 -- 10.20.193.27:0/213075927 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff01512f0 con 0x7feff01556c0 2026-02-13T02:22:03.776 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.773+0000 7feff4f456c0 1 -- 10.20.193.27:0/213075927 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feff0167ca0 con 0x7feff01556c0 2026-02-13T02:22:03.776 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.775+0000 7fefd77fe6c0 1 -- 10.20.193.27:0/213075927 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7fefe0003810 con 0x7feff01556c0 2026-02-13T02:22:03.777 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.775+0000 7fefd77fe6c0 1 --2- 10.20.193.27:0/213075927 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fefc80b5600 0x7fefc80d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:03.777 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.775+0000 7fefee5756c0 1 --2- 10.20.193.27:0/213075927 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fefc80b5600 0x7fefc80d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:03.777 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.775+0000 7fefd77fe6c0 1 -- 10.20.193.27:0/213075927 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6904+0+0 (secure 0 0 0) 0x7fefe00e45d0 con 0x7feff01556c0 2026-02-13T02:22:03.777 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.775+0000 7fefee5756c0 1 --2- 10.20.193.27:0/213075927 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fefc80b5600 0x7fefc80d5ab0 secure :-1 s=READY pgs=147 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fefe4004750 tx=0x7fefe4005f10 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:03.778 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.776+0000 7fefd77fe6c0 1 -- 10.20.193.27:0/213075927 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fefe00a74b0 con 0x7feff01556c0 2026-02-13T02:22:03.895 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.892+0000 7feff4f456c0 1 -- 10.20.193.27:0/213075927 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feff0162670 con 0x7feff01556c0 2026-02-13T02:22:03.895 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.892+0000 7fefd77fe6c0 1 -- 10.20.193.27:0/213075927 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2263 (secure 0 0 0) 0x7fefe00aa660 con 0x7feff01556c0 2026-02-13T02:22:03.895 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:22:03.895 INFO:teuthology.orchestra.run.trial027.stdout:{"epoch":3,"btime":"2026-02-13T02:22:03:514804+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.107:6833/3112510413","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6832","nonce":3112510413},{"type":"v1","addr":"10.20.193.107:6833","nonce":3112510413}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6833/2774006818","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6832","nonce":2774006818},{"type":"v1","addr":"10.20.193.28:6833","nonce":2774006818}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2026-02-13T02:22:03.896 INFO:teuthology.orchestra.run.trial027.stderr:dumped fsmap epoch 3 2026-02-13T02:22:03.897 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.894+0000 7feff4f456c0 1 -- 10.20.193.27:0/213075927 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fefc80b5600 msgr2=0x7fefc80d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:03.897 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.894+0000 7feff4f456c0 1 --2- 10.20.193.27:0/213075927 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fefc80b5600 0x7fefc80d5ab0 secure :-1 s=READY pgs=147 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fefe4004750 tx=0x7fefe4005f10 comp rx=0 tx=0).stop 2026-02-13T02:22:03.897 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.894+0000 7feff4f456c0 1 -- 10.20.193.27:0/213075927 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7feff01556c0 msgr2=0x7feff0166e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:03.897 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.894+0000 7feff4f456c0 1 --2- 10.20.193.27:0/213075927 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7feff01556c0 0x7feff0166e20 secure :-1 s=READY pgs=196 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fefe000a2c0 tx=0x7fefe000cbb0 comp rx=0 tx=0).stop 2026-02-13T02:22:03.897 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.894+0000 7feff4f456c0 1 -- 10.20.193.27:0/213075927 shutdown_connections 2026-02-13T02:22:03.897 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.894+0000 7feff4f456c0 1 --2- 10.20.193.27:0/213075927 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fefc80b5600 0x7fefc80d5ab0 unknown :-1 s=CLOSED pgs=147 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:03.897 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.894+0000 7feff4f456c0 1 --2- 10.20.193.27:0/213075927 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7feff01556c0 0x7feff0166e20 unknown :-1 s=CLOSED pgs=196 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:03.897 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.894+0000 7feff4f456c0 1 --2- 10.20.193.27:0/213075927 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7feff0061050 0x7feff0161a60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:03.897 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.894+0000 7feff4f456c0 1 --2- 10.20.193.27:0/213075927 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7feff0060740 0x7feff0161520 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:03.897 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.894+0000 7feff4f456c0 1 -- 10.20.193.27:0/213075927 >> 10.20.193.27:0/213075927 conn(0x7feff00ea8b0 msgr2=0x7feff010ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:03.898 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.894+0000 7feff4f456c0 1 -- 10.20.193.27:0/213075927 shutdown_connections 2026-02-13T02:22:03.898 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:03.894+0000 7feff4f456c0 1 -- 10.20.193.27:0/213075927 wait complete. 2026-02-13T02:22:04.151 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:03 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:04.151 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:03 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:04.151 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:03 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:04.152 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:03 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:04.152 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:03 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:04.152 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:03 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:04.152 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:03 trial027 ceph-mon[24567]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:04.152 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:03 trial027 ceph-mon[24567]: mds.? [v2:10.20.193.28:6832/2774006818,v1:10.20.193.28:6833/2774006818] up:boot 2026-02-13T02:22:04.152 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:03 trial027 ceph-mon[24567]: fsmap 2 up:standby 2026-02-13T02:22:04.152 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:03 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2026-02-13T02:22:04.152 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:03 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:04.152 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:03 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:04.152 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:03 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:04.152 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:03 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:04.152 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:03 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:04.153 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:03 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:04.153 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:03 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/213075927' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-13T02:22:04.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:03 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:04.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:03 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:04.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:03 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:04.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:03 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:04.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:03 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:04.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:03 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:04.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:03 trial028 ceph-mon[26701]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:04.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:03 trial028 ceph-mon[26701]: mds.? [v2:10.20.193.28:6832/2774006818,v1:10.20.193.28:6833/2774006818] up:boot 2026-02-13T02:22:04.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:03 trial028 ceph-mon[26701]: fsmap 2 up:standby 2026-02-13T02:22:04.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:03 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2026-02-13T02:22:04.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:03 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:04.245 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:03 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:04.245 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:03 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:04.245 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:03 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:04.245 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:03 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:04.245 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:03 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:04.245 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:03 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/213075927' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-13T02:22:04.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:03 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:04.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:03 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:04.250 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:03 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:04.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:03 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:04.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:03 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:04.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:03 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:04.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:03 trial107 ceph-mon[27422]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:04.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:03 trial107 ceph-mon[27422]: mds.? [v2:10.20.193.28:6832/2774006818,v1:10.20.193.28:6833/2774006818] up:boot 2026-02-13T02:22:04.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:03 trial107 ceph-mon[27422]: fsmap 2 up:standby 2026-02-13T02:22:04.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:03 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2026-02-13T02:22:04.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:03 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:04.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:03 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:04.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:03 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:04.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:03 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:04.252 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:03 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:04.252 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:03 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:04.252 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:03 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/213075927' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-13T02:22:04.909 INFO:journalctl@ceph.mds.d.trial027.stdout:Feb 13 02:22:04 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-d[60835]: starting mds.d at 2026-02-13T02:22:05.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:05 trial028 ceph-mon[26701]: mds.? [v2:10.20.193.27:6834/3709761799,v1:10.20.193.27:6835/3709761799] up:boot 2026-02-13T02:22:05.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:05 trial028 ceph-mon[26701]: fsmap 3 up:standby 2026-02-13T02:22:05.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:05 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2026-02-13T02:22:05.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:05 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:05.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:05 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:05.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:05 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:05.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:05 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:05.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:05 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:05.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:05 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:05.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:05 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:05.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:05 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:05.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:05 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:05.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:05 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:05.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:05 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:05.745 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:05 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:05.781 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:05 trial107 ceph-mon[27422]: mds.? [v2:10.20.193.27:6834/3709761799,v1:10.20.193.27:6835/3709761799] up:boot 2026-02-13T02:22:05.781 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:05 trial107 ceph-mon[27422]: fsmap 3 up:standby 2026-02-13T02:22:05.781 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:05 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2026-02-13T02:22:05.782 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:05 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:05.782 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:05 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:05.782 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:05 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:05.782 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:05 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:05.782 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:05 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:05.782 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:05 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:05.782 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:05 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:05.782 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:05 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:05.782 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:05 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:05.782 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:05 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:05.783 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:05 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:05.783 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:05 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:05.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:05 trial027 ceph-mon[24567]: mds.? [v2:10.20.193.27:6834/3709761799,v1:10.20.193.27:6835/3709761799] up:boot 2026-02-13T02:22:05.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:05 trial027 ceph-mon[24567]: fsmap 3 up:standby 2026-02-13T02:22:05.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:05 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2026-02-13T02:22:05.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:05 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:05.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:05 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:05.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:05 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:05.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:05 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:05.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:05 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:05.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:05 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:05.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:05 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:05.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:05 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:05.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:05 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:05.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:05 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:05.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:05 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:05.910 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:05 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:05.940 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph fs dump --format=json 2026-02-13T02:22:06.058 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:06.182 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.178+0000 7f7db07596c0 1 -- 10.20.193.27:0/3199044270 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7da81556e0 msgr2=0x7f7da8175ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:06.182 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.178+0000 7f7db07596c0 1 --2- 10.20.193.27:0/3199044270 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7da81556e0 0x7f7da8175ac0 secure :-1 s=READY pgs=199 gs=1 cs=0 l=1 c_cookie=5810113092533061068 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7da400aa40 tx=0x7f7da402fde0 comp rx=0 tx=0).stop 2026-02-13T02:22:06.182 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.178+0000 7f7db07596c0 1 -- 10.20.193.27:0/3199044270 shutdown_connections 2026-02-13T02:22:06.182 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.178+0000 7f7db07596c0 1 --2- 10.20.193.27:0/3199044270 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7da81556e0 0x7f7da8175ac0 unknown :-1 s=CLOSED pgs=199 gs=1 cs=0 l=1 c_cookie=5810113092533061068 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:06.182 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.178+0000 7f7db07596c0 1 --2- 10.20.193.27:0/3199044270 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f7da8061030 0x7f7da81551a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:06.182 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.178+0000 7f7db07596c0 1 --2- 10.20.193.27:0/3199044270 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7da8060720 0x7f7da8060af0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:06.182 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.178+0000 7f7db07596c0 1 -- 10.20.193.27:0/3199044270 >> 10.20.193.27:0/3199044270 conn(0x7f7da80ea8b0 msgr2=0x7f7da810acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:06.182 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.178+0000 7f7db07596c0 1 -- 10.20.193.27:0/3199044270 shutdown_connections 2026-02-13T02:22:06.182 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.178+0000 7f7db07596c0 1 -- 10.20.193.27:0/3199044270 wait complete. 2026-02-13T02:22:06.182 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.179+0000 7f7db07596c0 1 Processor -- start 2026-02-13T02:22:06.182 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.179+0000 7f7db07596c0 1 -- start start 2026-02-13T02:22:06.183 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.179+0000 7f7db07596c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7da8060720 0x7f7da8161530 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:06.183 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.179+0000 7f7db07596c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7da8061030 0x7f7da8161a70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:06.183 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.179+0000 7f7db07596c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f7da81556e0 0x7f7da8166e30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:06.183 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.179+0000 7f7db07596c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7da8167480 con 0x7f7da8061030 2026-02-13T02:22:06.183 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.179+0000 7f7db07596c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7da81675f0 con 0x7f7da8060720 2026-02-13T02:22:06.183 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.179+0000 7f7db07596c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7da81677b0 con 0x7f7da81556e0 2026-02-13T02:22:06.183 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.179+0000 7f7dadcc96c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7da8061030 0x7f7da8161a70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:06.183 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.179+0000 7f7dadcc96c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7da8061030 0x7f7da8161a70 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:54254/0 (socket says 10.20.193.27:54254) 2026-02-13T02:22:06.183 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.179+0000 7f7dadcc96c0 1 -- 10.20.193.27:0/1655818046 learned_addr learned my addr 10.20.193.27:0/1655818046 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:06.183 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.179+0000 7f7dae4ca6c0 1 --2- 10.20.193.27:0/1655818046 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7da8060720 0x7f7da8161530 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:06.183 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.179+0000 7f7dadcc96c0 1 -- 10.20.193.27:0/1655818046 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f7da81556e0 msgr2=0x7f7da8166e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:06.183 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.179+0000 7f7dadcc96c0 1 --2- 10.20.193.27:0/1655818046 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f7da81556e0 0x7f7da8166e30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:06.183 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.179+0000 7f7dadcc96c0 1 -- 10.20.193.27:0/1655818046 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7da8060720 msgr2=0x7f7da8161530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:06.183 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.179+0000 7f7dadcc96c0 1 --2- 10.20.193.27:0/1655818046 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7da8060720 0x7f7da8161530 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:06.183 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.179+0000 7f7dadcc96c0 1 -- 10.20.193.27:0/1655818046 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da400a660 con 0x7f7da8061030 2026-02-13T02:22:06.183 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.179+0000 7f7dadcc96c0 1 --2- 10.20.193.27:0/1655818046 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7da8061030 0x7f7da8161a70 secure :-1 s=READY pgs=200 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7d9c005dc0 tx=0x7f7d9c00bbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:06.183 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.179+0000 7f7d8f7fe6c0 1 -- 10.20.193.27:0/1655818046 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f7d9c004250 con 0x7f7da8061030 2026-02-13T02:22:06.183 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.179+0000 7f7d8f7fe6c0 1 -- 10.20.193.27:0/1655818046 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f7d9c009070 con 0x7f7da8061030 2026-02-13T02:22:06.183 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.179+0000 7f7db07596c0 1 -- 10.20.193.27:0/1655818046 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da8167a90 con 0x7f7da8061030 2026-02-13T02:22:06.183 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.179+0000 7f7d8f7fe6c0 1 -- 10.20.193.27:0/1655818046 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f7d9c00ce10 con 0x7f7da8061030 2026-02-13T02:22:06.183 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.179+0000 7f7db07596c0 1 -- 10.20.193.27:0/1655818046 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da8151310 con 0x7f7da8061030 2026-02-13T02:22:06.184 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.180+0000 7f7db07596c0 1 -- 10.20.193.27:0/1655818046 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7da8061c30 con 0x7f7da8061030 2026-02-13T02:22:06.184 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.181+0000 7f7d8f7fe6c0 1 -- 10.20.193.27:0/1655818046 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7f7d9c0028c0 con 0x7f7da8061030 2026-02-13T02:22:06.184 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.182+0000 7f7d8f7fe6c0 1 --2- 10.20.193.27:0/1655818046 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f7d840b5680 0x7f7d840d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:06.184 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.182+0000 7f7dae4ca6c0 1 --2- 10.20.193.27:0/1655818046 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f7d840b5680 0x7f7d840d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:06.184 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.182+0000 7f7d8f7fe6c0 1 -- 10.20.193.27:0/1655818046 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6904+0+0 (secure 0 0 0) 0x7f7d9c0e3580 con 0x7f7da8061030 2026-02-13T02:22:06.184 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.182+0000 7f7dae4ca6c0 1 --2- 10.20.193.27:0/1655818046 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f7d840b5680 0x7f7d840d5b30 secure :-1 s=READY pgs=151 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7d98005980 tx=0x7f7d98005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:06.184 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.182+0000 7f7d8f7fe6c0 1 -- 10.20.193.27:0/1655818046 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f7d9c0a9ec0 con 0x7f7da8061030 2026-02-13T02:22:06.300 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.298+0000 7f7db07596c0 1 -- 10.20.193.27:0/1655818046 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7da8060af0 con 0x7f7da8061030 2026-02-13T02:22:06.301 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:22:06.301 INFO:teuthology.orchestra.run.trial027.stdout:{"epoch":5,"btime":"2026-02-13T02:22:05:516781+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.27:6835/3709761799","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6834","nonce":3709761799},{"type":"v1","addr":"10.20.193.27:6835","nonce":3709761799}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.27:6837/747672540","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6836","nonce":747672540},{"type":"v1","addr":"10.20.193.27:6837","nonce":747672540}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.107:6833/3112510413","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6832","nonce":3112510413},{"type":"v1","addr":"10.20.193.107:6833","nonce":3112510413}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6833/2774006818","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6832","nonce":2774006818},{"type":"v1","addr":"10.20.193.28:6833","nonce":2774006818}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/1510901141","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":1510901141},{"type":"v1","addr":"10.20.193.28:6835","nonce":1510901141}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2026-02-13T02:22:06.301 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.299+0000 7f7d8f7fe6c0 1 -- 10.20.193.27:0/1655818046 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+4735 (secure 0 0 0) 0x7f7d9c070520 con 0x7f7da8061030 2026-02-13T02:22:06.301 INFO:teuthology.orchestra.run.trial027.stderr:dumped fsmap epoch 5 2026-02-13T02:22:06.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.300+0000 7f7db07596c0 1 -- 10.20.193.27:0/1655818046 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f7d840b5680 msgr2=0x7f7d840d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:06.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.300+0000 7f7db07596c0 1 --2- 10.20.193.27:0/1655818046 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f7d840b5680 0x7f7d840d5b30 secure :-1 s=READY pgs=151 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7d98005980 tx=0x7f7d98005910 comp rx=0 tx=0).stop 2026-02-13T02:22:06.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.300+0000 7f7db07596c0 1 -- 10.20.193.27:0/1655818046 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7da8061030 msgr2=0x7f7da8161a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:06.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.300+0000 7f7db07596c0 1 --2- 10.20.193.27:0/1655818046 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7da8061030 0x7f7da8161a70 secure :-1 s=READY pgs=200 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7d9c005dc0 tx=0x7f7d9c00bbd0 comp rx=0 tx=0).stop 2026-02-13T02:22:06.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.300+0000 7f7db07596c0 1 -- 10.20.193.27:0/1655818046 shutdown_connections 2026-02-13T02:22:06.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.300+0000 7f7db07596c0 1 --2- 10.20.193.27:0/1655818046 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f7d840b5680 0x7f7d840d5b30 unknown :-1 s=CLOSED pgs=151 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:06.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.300+0000 7f7db07596c0 1 --2- 10.20.193.27:0/1655818046 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f7da81556e0 0x7f7da8166e30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:06.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.300+0000 7f7db07596c0 1 --2- 10.20.193.27:0/1655818046 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7da8061030 0x7f7da8161a70 unknown :-1 s=CLOSED pgs=200 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:06.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.300+0000 7f7db07596c0 1 --2- 10.20.193.27:0/1655818046 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7da8060720 0x7f7da8161530 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:06.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.300+0000 7f7db07596c0 1 -- 10.20.193.27:0/1655818046 >> 10.20.193.27:0/1655818046 conn(0x7f7da80ea8b0 msgr2=0x7f7da810ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:06.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.300+0000 7f7db07596c0 1 -- 10.20.193.27:0/1655818046 shutdown_connections 2026-02-13T02:22:06.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:06.300+0000 7f7db07596c0 1 -- 10.20.193.27:0/1655818046 wait complete. 2026-02-13T02:22:06.501 INFO:journalctl@ceph.mds.f.trial107.stdout:Feb 13 02:22:06 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-f[55648]: starting mds.f at 2026-02-13T02:22:06.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:06 trial027 ceph-mon[24567]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:06.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:06 trial027 ceph-mon[24567]: mds.? [v2:10.20.193.27:6836/747672540,v1:10.20.193.27:6837/747672540] up:boot 2026-02-13T02:22:06.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:06 trial027 ceph-mon[24567]: mds.? [v2:10.20.193.28:6834/1510901141,v1:10.20.193.28:6835/1510901141] up:boot 2026-02-13T02:22:06.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:06 trial027 ceph-mon[24567]: fsmap 5 up:standby 2026-02-13T02:22:06.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:06 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2026-02-13T02:22:06.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:06 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2026-02-13T02:22:06.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:06 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:06.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:06 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:06.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:06 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:06.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:06 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:06.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:06 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:06.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:06 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:06.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:06 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1655818046' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-13T02:22:06.825 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:06 trial107 ceph-mon[27422]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:06.825 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:06 trial107 ceph-mon[27422]: mds.? [v2:10.20.193.27:6836/747672540,v1:10.20.193.27:6837/747672540] up:boot 2026-02-13T02:22:06.825 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:06 trial107 ceph-mon[27422]: mds.? [v2:10.20.193.28:6834/1510901141,v1:10.20.193.28:6835/1510901141] up:boot 2026-02-13T02:22:06.825 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:06 trial107 ceph-mon[27422]: fsmap 5 up:standby 2026-02-13T02:22:06.826 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:06 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2026-02-13T02:22:06.826 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:06 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2026-02-13T02:22:06.826 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:06 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:06.826 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:06 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:06.826 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:06 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:06.826 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:06 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:06.826 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:06 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:06.826 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:06 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:06.826 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:06 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1655818046' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-13T02:22:06.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:06 trial028 ceph-mon[26701]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:06.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:06 trial028 ceph-mon[26701]: mds.? [v2:10.20.193.27:6836/747672540,v1:10.20.193.27:6837/747672540] up:boot 2026-02-13T02:22:06.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:06 trial028 ceph-mon[26701]: mds.? [v2:10.20.193.28:6834/1510901141,v1:10.20.193.28:6835/1510901141] up:boot 2026-02-13T02:22:06.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:06 trial028 ceph-mon[26701]: fsmap 5 up:standby 2026-02-13T02:22:06.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:06 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2026-02-13T02:22:06.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:06 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2026-02-13T02:22:06.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:06 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:06.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:06 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:06.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:06 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:06.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:06 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:06.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:06 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:06.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:06 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:06.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:06 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1655818046' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-13T02:22:07.077 INFO:journalctl@ceph.mds.c.trial107.stdout:Feb 13 02:22:07 trial107 systemd[1]: Started Ceph mds.c for 707dbb29-0882-11f1-9262-d404e6e7d460. 2026-02-13T02:22:07.500 INFO:journalctl@ceph.mds.c.trial107.stdout:Feb 13 02:22:07 trial107 ceph-mds[55969]: -- 10.20.193.107:0/135287292 <== mon.1 v2:10.20.193.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562414939860 con 0x5624149ef400 2026-02-13T02:22:07.501 INFO:journalctl@ceph.mds.c.trial107.stdout:Feb 13 02:22:07 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-c[55965]: starting mds.c at 2026-02-13T02:22:07.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:07 trial028 ceph-mon[26701]: mds.? [v2:10.20.193.107:6834/3281057176,v1:10.20.193.107:6835/3281057176] up:boot 2026-02-13T02:22:07.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:07 trial028 ceph-mon[26701]: fsmap 6 up:standby 2026-02-13T02:22:07.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:07 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2026-02-13T02:22:07.743 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:07 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:07.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:07 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:07.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:07 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:07.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:07 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:07.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:07 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:07.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:07 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:07.744 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:07 trial028 ceph-mon[26701]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:07.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:07 trial027 ceph-mon[24567]: mds.? [v2:10.20.193.107:6834/3281057176,v1:10.20.193.107:6835/3281057176] up:boot 2026-02-13T02:22:07.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:07 trial027 ceph-mon[24567]: fsmap 6 up:standby 2026-02-13T02:22:07.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:07 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2026-02-13T02:22:07.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:07 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:07.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:07 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:07.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:07 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:07.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:07 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:07.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:07 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:07.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:07 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:07.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:07 trial027 ceph-mon[24567]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:08.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:07 trial107 ceph-mon[27422]: mds.? [v2:10.20.193.107:6834/3281057176,v1:10.20.193.107:6835/3281057176] up:boot 2026-02-13T02:22:08.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:07 trial107 ceph-mon[27422]: fsmap 6 up:standby 2026-02-13T02:22:08.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:07 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2026-02-13T02:22:08.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:07 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:08.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:07 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:08.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:07 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:08.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:07 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:08.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:07 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:08.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:07 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:08.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:07 trial107 ceph-mon[27422]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:08.243 INFO:journalctl@ceph.mds.h.trial028.stdout:Feb 13 02:22:07 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-h[54547]: starting mds.h at 2026-02-13T02:22:08.345 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph fs dump --format=json 2026-02-13T02:22:08.466 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:08.579 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.575+0000 7f8cae2486c0 1 -- 10.20.193.27:0/3867013238 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8ca810f230 msgr2=0x7f8ca810f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:08.580 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.575+0000 7f8cae2486c0 1 --2- 10.20.193.27:0/3867013238 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8ca810f230 0x7f8ca810f600 secure :-1 s=READY pgs=201 gs=1 cs=0 l=1 c_cookie=10462751944601730290 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8c940099a0 tx=0x7f8c9402ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:08.580 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.575+0000 7f8cae2486c0 1 -- 10.20.193.27:0/3867013238 shutdown_connections 2026-02-13T02:22:08.580 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.575+0000 7f8cae2486c0 1 --2- 10.20.193.27:0/3867013238 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8ca8115440 0x7f8ca8175de0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:08.580 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.575+0000 7f8cae2486c0 1 --2- 10.20.193.27:0/3867013238 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8ca810fc20 0x7f8ca8114c60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:08.580 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.575+0000 7f8cae2486c0 1 --2- 10.20.193.27:0/3867013238 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8ca810f230 0x7f8ca810f600 unknown :-1 s=CLOSED pgs=201 gs=1 cs=0 l=1 c_cookie=10462751944601730290 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:08.580 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.575+0000 7f8cae2486c0 1 -- 10.20.193.27:0/3867013238 >> 10.20.193.27:0/3867013238 conn(0x7f8ca80ecb30 msgr2=0x7f8ca810cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:08.580 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.575+0000 7f8cae2486c0 1 -- 10.20.193.27:0/3867013238 shutdown_connections 2026-02-13T02:22:08.580 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.575+0000 7f8cae2486c0 1 -- 10.20.193.27:0/3867013238 wait complete. 2026-02-13T02:22:08.581 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.576+0000 7f8cae2486c0 1 Processor -- start 2026-02-13T02:22:08.581 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.576+0000 7f8cae2486c0 1 -- start start 2026-02-13T02:22:08.581 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.576+0000 7f8cae2486c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8ca810f230 0x7f8ca811d320 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:08.581 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.576+0000 7f8cae2486c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8ca8115440 0x7f8ca811d860 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:08.581 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.576+0000 7f8cae2486c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8ca8122c20 0x7f8ca81230c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:08.581 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.576+0000 7f8cae2486c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca8123630 con 0x7f8ca8115440 2026-02-13T02:22:08.581 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.576+0000 7f8cae2486c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca81237a0 con 0x7f8ca810f230 2026-02-13T02:22:08.581 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.576+0000 7f8cae2486c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca8123960 con 0x7f8ca8122c20 2026-02-13T02:22:08.581 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.576+0000 7f8ca6ffd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8ca8115440 0x7f8ca811d860 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:08.581 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.576+0000 7f8ca6ffd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8ca8115440 0x7f8ca811d860 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:37530/0 (socket says 10.20.193.27:37530) 2026-02-13T02:22:08.581 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.576+0000 7f8ca6ffd6c0 1 -- 10.20.193.27:0/2834227799 learned_addr learned my addr 10.20.193.27:0/2834227799 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:08.582 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.576+0000 7f8ca77fe6c0 1 --2- 10.20.193.27:0/2834227799 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8ca810f230 0x7f8ca811d320 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:08.582 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.576+0000 7f8ca7fff6c0 1 --2- 10.20.193.27:0/2834227799 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8ca8122c20 0x7f8ca81230c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:08.582 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.576+0000 7f8ca6ffd6c0 1 -- 10.20.193.27:0/2834227799 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8ca8122c20 msgr2=0x7f8ca81230c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:08.582 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.576+0000 7f8ca6ffd6c0 1 --2- 10.20.193.27:0/2834227799 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8ca8122c20 0x7f8ca81230c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:08.582 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.576+0000 7f8ca6ffd6c0 1 -- 10.20.193.27:0/2834227799 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8ca810f230 msgr2=0x7f8ca811d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:08.582 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.576+0000 7f8ca6ffd6c0 1 --2- 10.20.193.27:0/2834227799 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8ca810f230 0x7f8ca811d320 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:08.582 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.576+0000 7f8ca6ffd6c0 1 -- 10.20.193.27:0/2834227799 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c94009590 con 0x7f8ca8115440 2026-02-13T02:22:08.582 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.576+0000 7f8ca6ffd6c0 1 --2- 10.20.193.27:0/2834227799 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8ca8115440 0x7f8ca811d860 secure :-1 s=READY pgs=202 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8c98005dc0 tx=0x7f8c9800bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:08.582 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.576+0000 7f8ca4ff96c0 1 -- 10.20.193.27:0/2834227799 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8c98004250 con 0x7f8ca8115440 2026-02-13T02:22:08.582 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.576+0000 7f8ca4ff96c0 1 -- 10.20.193.27:0/2834227799 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f8c98009070 con 0x7f8ca8115440 2026-02-13T02:22:08.582 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.576+0000 7f8cae2486c0 1 -- 10.20.193.27:0/2834227799 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca8163e10 con 0x7f8ca8115440 2026-02-13T02:22:08.582 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.576+0000 7f8ca4ff96c0 1 -- 10.20.193.27:0/2834227799 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8c9800cd50 con 0x7f8ca8115440 2026-02-13T02:22:08.583 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.576+0000 7f8cae2486c0 1 -- 10.20.193.27:0/2834227799 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca81642f0 con 0x7f8ca8115440 2026-02-13T02:22:08.583 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.577+0000 7f8cae2486c0 1 -- 10.20.193.27:0/2834227799 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ca8110500 con 0x7f8ca8115440 2026-02-13T02:22:08.584 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.578+0000 7f8ca4ff96c0 1 -- 10.20.193.27:0/2834227799 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7f8c980028c0 con 0x7f8ca8115440 2026-02-13T02:22:08.584 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.578+0000 7f8ca4ff96c0 1 --2- 10.20.193.27:0/2834227799 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8c7c0b5680 0x7f8c7c0d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:08.584 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.579+0000 7f8ca4ff96c0 1 -- 10.20.193.27:0/2834227799 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6904+0+0 (secure 0 0 0) 0x7f8c980e35e0 con 0x7f8ca8115440 2026-02-13T02:22:08.584 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.579+0000 7f8ca77fe6c0 1 --2- 10.20.193.27:0/2834227799 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8c7c0b5680 0x7f8c7c0d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:08.584 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.579+0000 7f8ca4ff96c0 1 -- 10.20.193.27:0/2834227799 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f8c980a64c0 con 0x7f8ca8115440 2026-02-13T02:22:08.584 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.579+0000 7f8ca77fe6c0 1 --2- 10.20.193.27:0/2834227799 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8c7c0b5680 0x7f8c7c0d5b30 secure :-1 s=READY pgs=154 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8c940092e0 tx=0x7f8c94005c00 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:08.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:08 trial027 ceph-mon[24567]: mds.? [v2:10.20.193.107:6836/2849542469,v1:10.20.193.107:6837/2849542469] up:boot 2026-02-13T02:22:08.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:08 trial027 ceph-mon[24567]: fsmap 7 up:standby 2026-02-13T02:22:08.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:08 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2026-02-13T02:22:08.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:08 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:08.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:08 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:08.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:08 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:08.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:08 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:08.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:08 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:08.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:08 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:08.695 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.692+0000 7f8cae2486c0 1 -- 10.20.193.27:0/2834227799 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ca810f600 con 0x7f8ca8115440 2026-02-13T02:22:08.695 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.693+0000 7f8ca4ff96c0 1 -- 10.20.193.27:0/2834227799 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+7213 (secure 0 0 0) 0x7f8c980a9670 con 0x7f8ca8115440 2026-02-13T02:22:08.696 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:22:08.696 INFO:teuthology.orchestra.run.trial027.stdout:{"epoch":8,"btime":"2026-02-13T02:22:08:605561+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.27:6835/3709761799","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6834","nonce":3709761799},{"type":"v1","addr":"10.20.193.27:6835","nonce":3709761799}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.27:6837/747672540","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6836","nonce":747672540},{"type":"v1","addr":"10.20.193.27:6837","nonce":747672540}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.107:6833/3112510413","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6832","nonce":3112510413},{"type":"v1","addr":"10.20.193.107:6833","nonce":3112510413}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6833/2774006818","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6832","nonce":2774006818},{"type":"v1","addr":"10.20.193.28:6833","nonce":2774006818}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.107:6835/3281057176","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6834","nonce":3281057176},{"type":"v1","addr":"10.20.193.107:6835","nonce":3281057176}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.107:6837/2849542469","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6836","nonce":2849542469},{"type":"v1","addr":"10.20.193.107:6837","nonce":2849542469}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/1510901141","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":1510901141},{"type":"v1","addr":"10.20.193.28:6835","nonce":1510901141}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/543345874","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":543345874},{"type":"v1","addr":"10.20.193.28:6837","nonce":543345874}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2026-02-13T02:22:08.697 INFO:teuthology.orchestra.run.trial027.stderr:dumped fsmap epoch 8 2026-02-13T02:22:08.697 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.694+0000 7f8cae2486c0 1 -- 10.20.193.27:0/2834227799 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8c7c0b5680 msgr2=0x7f8c7c0d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:08.697 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.694+0000 7f8cae2486c0 1 --2- 10.20.193.27:0/2834227799 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8c7c0b5680 0x7f8c7c0d5b30 secure :-1 s=READY pgs=154 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8c940092e0 tx=0x7f8c94005c00 comp rx=0 tx=0).stop 2026-02-13T02:22:08.697 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.694+0000 7f8cae2486c0 1 -- 10.20.193.27:0/2834227799 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8ca8115440 msgr2=0x7f8ca811d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:08.697 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.694+0000 7f8cae2486c0 1 --2- 10.20.193.27:0/2834227799 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8ca8115440 0x7f8ca811d860 secure :-1 s=READY pgs=202 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8c98005dc0 tx=0x7f8c9800bbb0 comp rx=0 tx=0).stop 2026-02-13T02:22:08.697 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.694+0000 7f8cae2486c0 1 -- 10.20.193.27:0/2834227799 shutdown_connections 2026-02-13T02:22:08.697 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.694+0000 7f8cae2486c0 1 --2- 10.20.193.27:0/2834227799 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8c7c0b5680 0x7f8c7c0d5b30 unknown :-1 s=CLOSED pgs=154 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:08.697 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.694+0000 7f8cae2486c0 1 --2- 10.20.193.27:0/2834227799 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8ca8122c20 0x7f8ca81230c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:08.698 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.694+0000 7f8cae2486c0 1 --2- 10.20.193.27:0/2834227799 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8ca8115440 0x7f8ca811d860 unknown :-1 s=CLOSED pgs=202 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:08.698 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.694+0000 7f8cae2486c0 1 --2- 10.20.193.27:0/2834227799 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8ca810f230 0x7f8ca811d320 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:08.698 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.694+0000 7f8cae2486c0 1 -- 10.20.193.27:0/2834227799 >> 10.20.193.27:0/2834227799 conn(0x7f8ca80ecb30 msgr2=0x7f8ca810cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:08.698 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.694+0000 7f8cae2486c0 1 -- 10.20.193.27:0/2834227799 shutdown_connections 2026-02-13T02:22:08.698 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:08.694+0000 7f8cae2486c0 1 -- 10.20.193.27:0/2834227799 wait complete. 2026-02-13T02:22:08.993 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:08 trial028 ceph-mon[26701]: mds.? [v2:10.20.193.107:6836/2849542469,v1:10.20.193.107:6837/2849542469] up:boot 2026-02-13T02:22:08.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:08 trial028 ceph-mon[26701]: fsmap 7 up:standby 2026-02-13T02:22:08.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:08 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2026-02-13T02:22:08.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:08 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:08.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:08 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:08.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:08 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:08.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:08 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:08.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:08 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:08.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:08 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:09.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:08 trial107 ceph-mon[27422]: mds.? [v2:10.20.193.107:6836/2849542469,v1:10.20.193.107:6837/2849542469] up:boot 2026-02-13T02:22:09.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:08 trial107 ceph-mon[27422]: fsmap 7 up:standby 2026-02-13T02:22:09.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:08 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2026-02-13T02:22:09.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:08 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:09.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:08 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:09.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:08 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:09.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:08 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:09.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:08 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:09.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:08 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:09.002 INFO:journalctl@ceph.mds.i.trial107.stdout:Feb 13 02:22:08 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-i[56293]: starting mds.i at 2026-02-13T02:22:09.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:09 trial027 ceph-mon[24567]: mds.? [v2:10.20.193.28:6836/543345874,v1:10.20.193.28:6837/543345874] up:boot 2026-02-13T02:22:09.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:09 trial027 ceph-mon[24567]: fsmap 8 up:standby 2026-02-13T02:22:09.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:09 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2026-02-13T02:22:09.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:09 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2834227799' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-13T02:22:09.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:09 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:09.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:09 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:09.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:09 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:09.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:09 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:09.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:09 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:09.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:09 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:09.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:09 trial027 ceph-mon[24567]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:09.993 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:09 trial028 ceph-mon[26701]: mds.? [v2:10.20.193.28:6836/543345874,v1:10.20.193.28:6837/543345874] up:boot 2026-02-13T02:22:09.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:09 trial028 ceph-mon[26701]: fsmap 8 up:standby 2026-02-13T02:22:09.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:09 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2026-02-13T02:22:09.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:09 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2834227799' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-13T02:22:09.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:09 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:09.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:09 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:09.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:09 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:09.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:09 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:09.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:09 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:09.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:09 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:09.995 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:09 trial028 ceph-mon[26701]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:10.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:09 trial107 ceph-mon[27422]: mds.? [v2:10.20.193.28:6836/543345874,v1:10.20.193.28:6837/543345874] up:boot 2026-02-13T02:22:10.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:09 trial107 ceph-mon[27422]: fsmap 8 up:standby 2026-02-13T02:22:10.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:09 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2026-02-13T02:22:10.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:09 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2834227799' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-13T02:22:10.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:09 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:10.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:09 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:10.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:09 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:10.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:09 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:10.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:09 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:10.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:09 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:10.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:09 trial107 ceph-mon[27422]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:10.618 INFO:journalctl@ceph.mds.a.trial027.stdout:Feb 13 02:22:10 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-a[61693]: starting mds.a at 2026-02-13T02:22:10.754 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph fs dump --format=json 2026-02-13T02:22:10.873 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:10 trial028 ceph-mon[26701]: mds.? [v2:10.20.193.107:6838/3260223906,v1:10.20.193.107:6839/3260223906] up:boot 2026-02-13T02:22:10.873 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:10 trial028 ceph-mon[26701]: mds.? [v2:10.20.193.27:6838/761293857,v1:10.20.193.27:6839/761293857] up:boot 2026-02-13T02:22:10.874 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:10 trial028 ceph-mon[26701]: fsmap 10 up:standby 2026-02-13T02:22:10.874 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:10 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2026-02-13T02:22:10.874 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:10 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2026-02-13T02:22:10.874 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:10 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:10.874 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:10 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:10.874 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:10 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:10.874 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:10 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:10.874 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:10 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:10.874 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:10 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:10.874 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:10 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:10.874 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:10 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:10.874 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:10 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:10.875 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:10 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:10.875 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:10 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:10.875 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:10 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:10.875 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:10.897 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:10 trial027 ceph-mon[24567]: mds.? [v2:10.20.193.107:6838/3260223906,v1:10.20.193.107:6839/3260223906] up:boot 2026-02-13T02:22:10.897 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:10 trial027 ceph-mon[24567]: mds.? [v2:10.20.193.27:6838/761293857,v1:10.20.193.27:6839/761293857] up:boot 2026-02-13T02:22:10.897 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:10 trial027 ceph-mon[24567]: fsmap 10 up:standby 2026-02-13T02:22:10.897 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:10 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2026-02-13T02:22:10.897 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:10 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2026-02-13T02:22:10.897 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:10 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:10.898 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:10 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:10.898 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:10 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:10.898 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:10 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:10.898 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:10 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:10.898 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:10 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:10.898 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:10 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:10.898 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:10 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:10.898 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:10 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:10.898 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:10 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:10.898 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:10 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:10.898 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:10 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:10.997 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.992+0000 7f4256c516c0 1 -- 10.20.193.27:0/1201183747 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f425010f200 msgr2=0x7f425010f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:10.997 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.992+0000 7f4256c516c0 1 --2- 10.20.193.27:0/1201183747 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f425010f200 0x7f425010f5d0 secure :-1 s=READY pgs=207 gs=1 cs=0 l=1 c_cookie=13733272890405271238 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f42400099a0 tx=0x7f424002ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:10.997 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.992+0000 7f4256c516c0 1 -- 10.20.193.27:0/1201183747 shutdown_connections 2026-02-13T02:22:10.998 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.992+0000 7f4256c516c0 1 --2- 10.20.193.27:0/1201183747 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4250115410 0x7f4250175db0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:10.998 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.992+0000 7f4256c516c0 1 --2- 10.20.193.27:0/1201183747 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f425010fbf0 0x7f4250114c30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:10.998 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.992+0000 7f4256c516c0 1 --2- 10.20.193.27:0/1201183747 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f425010f200 0x7f425010f5d0 unknown :-1 s=CLOSED pgs=207 gs=1 cs=0 l=1 c_cookie=13733272890405271238 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:10.998 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.992+0000 7f4256c516c0 1 -- 10.20.193.27:0/1201183747 >> 10.20.193.27:0/1201183747 conn(0x7f42500ecb00 msgr2=0x7f425010cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:10.998 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.992+0000 7f4256c516c0 1 -- 10.20.193.27:0/1201183747 shutdown_connections 2026-02-13T02:22:10.998 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.992+0000 7f4256c516c0 1 -- 10.20.193.27:0/1201183747 wait complete. 2026-02-13T02:22:10.998 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.993+0000 7f4256c516c0 1 Processor -- start 2026-02-13T02:22:10.998 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.993+0000 7f4256c516c0 1 -- start start 2026-02-13T02:22:10.998 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.993+0000 7f4256c516c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f425010f200 0x7f425011d290 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:10.999 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.993+0000 7f4256c516c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4250115410 0x7f425011d7d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:10.999 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.993+0000 7f4256c516c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4250122b90 0x7f4250123030 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:10.999 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.993+0000 7f4256c516c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f42501235a0 con 0x7f4250115410 2026-02-13T02:22:10.999 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.993+0000 7f4256c516c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4250123710 con 0x7f4250122b90 2026-02-13T02:22:10.999 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.993+0000 7f4256c516c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f42501238d0 con 0x7f425010f200 2026-02-13T02:22:10.999 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.993+0000 7f4247fff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4250115410 0x7f425011d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:10.999 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.993+0000 7f4247fff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4250115410 0x7f425011d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:37590/0 (socket says 10.20.193.27:37590) 2026-02-13T02:22:10.999 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.993+0000 7f4247fff6c0 1 -- 10.20.193.27:0/3508430687 learned_addr learned my addr 10.20.193.27:0/3508430687 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:10.999 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.993+0000 7f42551c36c0 1 --2- 10.20.193.27:0/3508430687 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4250122b90 0x7f4250123030 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:10.999 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.993+0000 7f42549c26c0 1 --2- 10.20.193.27:0/3508430687 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f425010f200 0x7f425011d290 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:10.999 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.993+0000 7f4247fff6c0 1 -- 10.20.193.27:0/3508430687 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f425010f200 msgr2=0x7f425011d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:11.000 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.993+0000 7f4247fff6c0 1 --2- 10.20.193.27:0/3508430687 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f425010f200 0x7f425011d290 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:11.000 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.993+0000 7f4247fff6c0 1 -- 10.20.193.27:0/3508430687 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4250122b90 msgr2=0x7f4250123030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:11.000 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.993+0000 7f4247fff6c0 1 --2- 10.20.193.27:0/3508430687 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4250122b90 0x7f4250123030 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:11.000 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.993+0000 7f4247fff6c0 1 -- 10.20.193.27:0/3508430687 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4240009590 con 0x7f4250115410 2026-02-13T02:22:11.000 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.993+0000 7f42551c36c0 1 --2- 10.20.193.27:0/3508430687 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4250122b90 0x7f4250123030 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:22:11.000 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.993+0000 7f4247fff6c0 1 --2- 10.20.193.27:0/3508430687 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4250115410 0x7f425011d7d0 secure :-1 s=READY pgs=208 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f423800b6c0 tx=0x7f423800bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:11.000 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.993+0000 7f4245ffb6c0 1 -- 10.20.193.27:0/3508430687 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4238004250 con 0x7f4250115410 2026-02-13T02:22:11.000 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.993+0000 7f4245ffb6c0 1 -- 10.20.193.27:0/3508430687 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4238009070 con 0x7f4250115410 2026-02-13T02:22:11.000 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.993+0000 7f4256c516c0 1 -- 10.20.193.27:0/3508430687 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4250163d80 con 0x7f4250115410 2026-02-13T02:22:11.000 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.993+0000 7f4245ffb6c0 1 -- 10.20.193.27:0/3508430687 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f423800ccb0 con 0x7f4250115410 2026-02-13T02:22:11.000 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.993+0000 7f4256c516c0 1 -- 10.20.193.27:0/3508430687 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4250164210 con 0x7f4250115410 2026-02-13T02:22:11.000 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.994+0000 7f4256c516c0 1 -- 10.20.193.27:0/3508430687 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42501104d0 con 0x7f4250115410 2026-02-13T02:22:11.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:10 trial107 ceph-mon[27422]: mds.? [v2:10.20.193.107:6838/3260223906,v1:10.20.193.107:6839/3260223906] up:boot 2026-02-13T02:22:11.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:10 trial107 ceph-mon[27422]: mds.? [v2:10.20.193.27:6838/761293857,v1:10.20.193.27:6839/761293857] up:boot 2026-02-13T02:22:11.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:10 trial107 ceph-mon[27422]: fsmap 10 up:standby 2026-02-13T02:22:11.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:10 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2026-02-13T02:22:11.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:10 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2026-02-13T02:22:11.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:10 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:11.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:10 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:11.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:10 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:11.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:10 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:11.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:10 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:11.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:10 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:11.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:10 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:11.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:10 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:11.002 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:10 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:11.003 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:10 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-13T02:22:11.003 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:10 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-13T02:22:11.003 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:10 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:11.003 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.995+0000 7f4245ffb6c0 1 -- 10.20.193.27:0/3508430687 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7f42380028c0 con 0x7f4250115410 2026-02-13T02:22:11.004 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.996+0000 7f4245ffb6c0 1 --2- 10.20.193.27:0/3508430687 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f422c0b5680 0x7f422c0d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:11.004 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.996+0000 7f42549c26c0 1 --2- 10.20.193.27:0/3508430687 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f422c0b5680 0x7f422c0d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:11.004 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.996+0000 7f4245ffb6c0 1 -- 10.20.193.27:0/3508430687 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6904+0+0 (secure 0 0 0) 0x7f42380e3690 con 0x7f4250115410 2026-02-13T02:22:11.004 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.996+0000 7f4245ffb6c0 1 -- 10.20.193.27:0/3508430687 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f42380e3b10 con 0x7f4250115410 2026-02-13T02:22:11.004 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:10.996+0000 7f42549c26c0 1 --2- 10.20.193.27:0/3508430687 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f422c0b5680 0x7f422c0d5b30 secure :-1 s=READY pgs=159 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f42400023e0 tx=0x7f4240005de0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:11.112 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:11.110+0000 7f4256c516c0 1 -- 10.20.193.27:0/3508430687 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f425010f5d0 con 0x7f4250115410 2026-02-13T02:22:11.113 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:11.110+0000 7f4245ffb6c0 1 -- 10.20.193.27:0/3508430687 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+9690 (secure 0 0 0) 0x7f42380a6570 con 0x7f4250115410 2026-02-13T02:22:11.113 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:22:11.113 INFO:teuthology.orchestra.run.trial027.stdout:{"epoch":10,"btime":"2026-02-13T02:22:10:783417+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.27:6835/3709761799","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6834","nonce":3709761799},{"type":"v1","addr":"10.20.193.27:6835","nonce":3709761799}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.27:6837/747672540","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6836","nonce":747672540},{"type":"v1","addr":"10.20.193.27:6837","nonce":747672540}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14730,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.27:6839/761293857","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6838","nonce":761293857},{"type":"v1","addr":"10.20.193.27:6839","nonce":761293857}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":14736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.27:6841/1659204922","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6840","nonce":1659204922},{"type":"v1","addr":"10.20.193.27:6841","nonce":1659204922}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.107:6833/3112510413","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6832","nonce":3112510413},{"type":"v1","addr":"10.20.193.107:6833","nonce":3112510413}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6833/2774006818","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6832","nonce":2774006818},{"type":"v1","addr":"10.20.193.28:6833","nonce":2774006818}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.107:6835/3281057176","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6834","nonce":3281057176},{"type":"v1","addr":"10.20.193.107:6835","nonce":3281057176}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.107:6837/2849542469","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6836","nonce":2849542469},{"type":"v1","addr":"10.20.193.107:6837","nonce":2849542469}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/1510901141","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":1510901141},{"type":"v1","addr":"10.20.193.28:6835","nonce":1510901141}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.107:6839/3260223906","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6838","nonce":3260223906},{"type":"v1","addr":"10.20.193.107:6839","nonce":3260223906}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/543345874","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":543345874},{"type":"v1","addr":"10.20.193.28:6837","nonce":543345874}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2026-02-13T02:22:11.114 INFO:teuthology.orchestra.run.trial027.stderr:dumped fsmap epoch 10 2026-02-13T02:22:11.114 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:11.111+0000 7f4256c516c0 1 -- 10.20.193.27:0/3508430687 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f422c0b5680 msgr2=0x7f422c0d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:11.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:11.111+0000 7f4256c516c0 1 --2- 10.20.193.27:0/3508430687 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f422c0b5680 0x7f422c0d5b30 secure :-1 s=READY pgs=159 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f42400023e0 tx=0x7f4240005de0 comp rx=0 tx=0).stop 2026-02-13T02:22:11.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:11.111+0000 7f4256c516c0 1 -- 10.20.193.27:0/3508430687 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4250115410 msgr2=0x7f425011d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:11.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:11.111+0000 7f4256c516c0 1 --2- 10.20.193.27:0/3508430687 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4250115410 0x7f425011d7d0 secure :-1 s=READY pgs=208 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f423800b6c0 tx=0x7f423800bb90 comp rx=0 tx=0).stop 2026-02-13T02:22:11.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:11.111+0000 7f4256c516c0 1 -- 10.20.193.27:0/3508430687 shutdown_connections 2026-02-13T02:22:11.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:11.111+0000 7f4256c516c0 1 --2- 10.20.193.27:0/3508430687 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f422c0b5680 0x7f422c0d5b30 unknown :-1 s=CLOSED pgs=159 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:11.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:11.111+0000 7f4256c516c0 1 --2- 10.20.193.27:0/3508430687 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4250122b90 0x7f4250123030 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:11.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:11.111+0000 7f4256c516c0 1 --2- 10.20.193.27:0/3508430687 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4250115410 0x7f425011d7d0 unknown :-1 s=CLOSED pgs=208 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:11.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:11.111+0000 7f4256c516c0 1 --2- 10.20.193.27:0/3508430687 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f425010f200 0x7f425011d290 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:11.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:11.111+0000 7f4256c516c0 1 -- 10.20.193.27:0/3508430687 >> 10.20.193.27:0/3508430687 conn(0x7f42500ecb00 msgr2=0x7f425010cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:11.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:11.111+0000 7f4256c516c0 1 -- 10.20.193.27:0/3508430687 shutdown_connections 2026-02-13T02:22:11.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:11.111+0000 7f4256c516c0 1 -- 10.20.193.27:0/3508430687 wait complete. 2026-02-13T02:22:11.494 INFO:journalctl@ceph.mds.b.trial028.stdout:Feb 13 02:22:11 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-b[54870]: starting mds.b at 2026-02-13T02:22:11.993 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:11 trial028 ceph-mon[26701]: mds.? [v2:10.20.193.27:6840/1659204922,v1:10.20.193.27:6841/1659204922] up:boot 2026-02-13T02:22:11.993 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:11 trial028 ceph-mon[26701]: fsmap 11 up:standby 2026-02-13T02:22:11.993 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:11 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2026-02-13T02:22:11.993 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:11 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3508430687' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-13T02:22:11.993 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:11 trial028 ceph-mon[26701]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:11.993 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:11 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:11.993 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:11 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:11.993 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:11 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:11.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:11 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:11.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:11 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:11.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:11 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:22:11.994 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:11 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:12.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:11 trial107 ceph-mon[27422]: mds.? [v2:10.20.193.27:6840/1659204922,v1:10.20.193.27:6841/1659204922] up:boot 2026-02-13T02:22:12.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:11 trial107 ceph-mon[27422]: fsmap 11 up:standby 2026-02-13T02:22:12.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:11 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2026-02-13T02:22:12.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:11 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3508430687' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-13T02:22:12.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:11 trial107 ceph-mon[27422]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:12.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:11 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:12.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:11 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:12.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:11 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:12.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:11 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:12.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:11 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:12.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:11 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:22:12.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:11 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:12.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:11 trial027 ceph-mon[24567]: mds.? [v2:10.20.193.27:6840/1659204922,v1:10.20.193.27:6841/1659204922] up:boot 2026-02-13T02:22:12.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:11 trial027 ceph-mon[24567]: fsmap 11 up:standby 2026-02-13T02:22:12.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:11 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2026-02-13T02:22:12.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:11 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3508430687' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-13T02:22:12.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:11 trial027 ceph-mon[24567]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:12.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:11 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:12.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:11 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:12.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:11 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:12.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:11 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:12.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:11 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:12.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:11 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:22:12.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:11 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:13.092 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:12 trial107 ceph-mon[27422]: mds.? [v2:10.20.193.28:6838/2398862063,v1:10.20.193.28:6839/2398862063] up:boot 2026-02-13T02:22:13.092 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:12 trial107 ceph-mon[27422]: fsmap 12 up:standby 2026-02-13T02:22:13.092 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:12 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2026-02-13T02:22:13.092 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:12 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:13.092 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:12 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:13.092 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:12 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:13.092 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:12 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:13.092 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:12 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:13.092 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:12 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:13.092 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:12 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:13.092 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:12 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:22:13.092 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:12 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:13.092 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:12 trial107 ceph-mon[27422]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:13.092 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:12 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:22:13.092 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:12 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:22:13.092 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:12 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:13.156 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph fs dump --format=json 2026-02-13T02:22:13.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:12 trial027 ceph-mon[24567]: mds.? [v2:10.20.193.28:6838/2398862063,v1:10.20.193.28:6839/2398862063] up:boot 2026-02-13T02:22:13.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:12 trial027 ceph-mon[24567]: fsmap 12 up:standby 2026-02-13T02:22:13.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:12 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2026-02-13T02:22:13.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:12 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:13.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:12 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:13.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:12 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:13.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:12 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:13.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:12 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:13.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:12 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:13.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:12 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:13.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:12 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:22:13.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:12 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:13.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:12 trial027 ceph-mon[24567]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:13.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:12 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:22:13.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:12 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:22:13.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:12 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:13.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:12 trial028 ceph-mon[26701]: mds.? [v2:10.20.193.28:6838/2398862063,v1:10.20.193.28:6839/2398862063] up:boot 2026-02-13T02:22:13.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:12 trial028 ceph-mon[26701]: fsmap 12 up:standby 2026-02-13T02:22:13.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:12 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2026-02-13T02:22:13.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:12 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:13.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:12 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:13.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:12 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:13.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:12 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:13.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:12 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:13.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:12 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:13.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:12 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:13.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:12 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:22:13.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:12 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:13.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:12 trial028 ceph-mon[26701]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:13.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:12 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:22:13.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:12 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:22:13.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:12 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:13.275 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:13.395 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.392+0000 7f2a506896c0 1 -- 10.20.193.27:0/999763673 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2a481198f0 msgr2=0x7f2a4817a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:13.395 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.392+0000 7f2a506896c0 1 --2- 10.20.193.27:0/999763673 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2a481198f0 0x7f2a4817a2c0 secure :-1 s=READY pgs=209 gs=1 cs=0 l=1 c_cookie=4703760115387980116 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2a4400aa40 tx=0x7f2a4402fde0 comp rx=0 tx=0).stop 2026-02-13T02:22:13.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.392+0000 7f2a506896c0 1 -- 10.20.193.27:0/999763673 shutdown_connections 2026-02-13T02:22:13.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.392+0000 7f2a506896c0 1 --2- 10.20.193.27:0/999763673 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2a481198f0 0x7f2a4817a2c0 unknown :-1 s=CLOSED pgs=209 gs=1 cs=0 l=1 c_cookie=4703760115387980116 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:13.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.392+0000 7f2a506896c0 1 --2- 10.20.193.27:0/999763673 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2a48114190 0x7f2a48119140 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:13.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.392+0000 7f2a506896c0 1 --2- 10.20.193.27:0/999763673 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2a481137a0 0x7f2a48113b70 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:13.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.392+0000 7f2a506896c0 1 -- 10.20.193.27:0/999763673 >> 10.20.193.27:0/999763673 conn(0x7f2a48060760 msgr2=0x7f2a48060d00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:13.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.392+0000 7f2a506896c0 1 -- 10.20.193.27:0/999763673 shutdown_connections 2026-02-13T02:22:13.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.392+0000 7f2a506896c0 1 -- 10.20.193.27:0/999763673 wait complete. 2026-02-13T02:22:13.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.393+0000 7f2a506896c0 1 Processor -- start 2026-02-13T02:22:13.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.393+0000 7f2a506896c0 1 -- start start 2026-02-13T02:22:13.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.393+0000 7f2a506896c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2a481137a0 0x7f2a48121710 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:13.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.393+0000 7f2a506896c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2a481198f0 0x7f2a48121c50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:13.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.393+0000 7f2a506896c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2a48127010 0x7f2a481274b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:13.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.393+0000 7f2a506896c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2a48127a20 con 0x7f2a48127010 2026-02-13T02:22:13.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.393+0000 7f2a506896c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2a48127b90 con 0x7f2a481198f0 2026-02-13T02:22:13.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.393+0000 7f2a506896c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2a48127d50 con 0x7f2a481137a0 2026-02-13T02:22:13.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.393+0000 7f2a4ebfb6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2a48127010 0x7f2a481274b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:13.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.393+0000 7f2a4ebfb6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2a48127010 0x7f2a481274b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:37606/0 (socket says 10.20.193.27:37606) 2026-02-13T02:22:13.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.393+0000 7f2a4ebfb6c0 1 -- 10.20.193.27:0/1599540744 learned_addr learned my addr 10.20.193.27:0/1599540744 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:13.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.393+0000 7f2a4ebfb6c0 1 -- 10.20.193.27:0/1599540744 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2a481137a0 msgr2=0x7f2a48121710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:13.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.393+0000 7f2a4ebfb6c0 1 --2- 10.20.193.27:0/1599540744 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2a481137a0 0x7f2a48121710 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:13.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.393+0000 7f2a4ebfb6c0 1 -- 10.20.193.27:0/1599540744 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2a481198f0 msgr2=0x7f2a48121c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:13.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.393+0000 7f2a4ebfb6c0 1 --2- 10.20.193.27:0/1599540744 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2a481198f0 0x7f2a48121c50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:13.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.393+0000 7f2a4ebfb6c0 1 -- 10.20.193.27:0/1599540744 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a4400a660 con 0x7f2a48127010 2026-02-13T02:22:13.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.393+0000 7f2a4ebfb6c0 1 --2- 10.20.193.27:0/1599540744 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2a48127010 0x7f2a481274b0 secure :-1 s=READY pgs=210 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2a4400a390 tx=0x7f2a44005530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:13.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.393+0000 7f2a2f7fe6c0 1 -- 10.20.193.27:0/1599540744 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2a44003a30 con 0x7f2a48127010 2026-02-13T02:22:13.397 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.393+0000 7f2a506896c0 1 -- 10.20.193.27:0/1599540744 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a481681a0 con 0x7f2a48127010 2026-02-13T02:22:13.397 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.393+0000 7f2a2f7fe6c0 1 -- 10.20.193.27:0/1599540744 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f2a44003b90 con 0x7f2a48127010 2026-02-13T02:22:13.397 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.393+0000 7f2a2f7fe6c0 1 -- 10.20.193.27:0/1599540744 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2a440382f0 con 0x7f2a48127010 2026-02-13T02:22:13.397 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.393+0000 7f2a506896c0 1 -- 10.20.193.27:0/1599540744 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a481685d0 con 0x7f2a48127010 2026-02-13T02:22:13.397 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.394+0000 7f2a506896c0 1 -- 10.20.193.27:0/1599540744 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a481149e0 con 0x7f2a48127010 2026-02-13T02:22:13.397 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.395+0000 7f2a2f7fe6c0 1 -- 10.20.193.27:0/1599540744 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7f2a44008260 con 0x7f2a48127010 2026-02-13T02:22:13.398 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.396+0000 7f2a2f7fe6c0 1 --2- 10.20.193.27:0/1599540744 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f2a240b5680 0x7f2a240d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:13.398 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.396+0000 7f2a2f7fe6c0 1 -- 10.20.193.27:0/1599540744 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6904+0+0 (secure 0 0 0) 0x7f2a44108bb0 con 0x7f2a48127010 2026-02-13T02:22:13.398 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.396+0000 7f2a4e3fa6c0 1 --2- 10.20.193.27:0/1599540744 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f2a240b5680 0x7f2a240d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:13.399 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.396+0000 7f2a2f7fe6c0 1 -- 10.20.193.27:0/1599540744 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f2a44141680 con 0x7f2a48127010 2026-02-13T02:22:13.399 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.396+0000 7f2a4e3fa6c0 1 --2- 10.20.193.27:0/1599540744 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f2a240b5680 0x7f2a240d5b30 secure :-1 s=READY pgs=161 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2a38005980 tx=0x7f2a38005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:13.512 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.510+0000 7f2a506896c0 1 -- 10.20.193.27:0/1599540744 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a48114190 con 0x7f2a48127010 2026-02-13T02:22:13.513 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.510+0000 7f2a2f7fe6c0 1 -- 10.20.193.27:0/1599540744 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+10516 (secure 0 0 0) 0x7f2a440cbad0 con 0x7f2a48127010 2026-02-13T02:22:13.514 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:22:13.514 INFO:teuthology.orchestra.run.trial027.stdout:{"epoch":11,"btime":"2026-02-13T02:22:11:784629+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.27:6835/3709761799","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6834","nonce":3709761799},{"type":"v1","addr":"10.20.193.27:6835","nonce":3709761799}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.27:6837/747672540","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6836","nonce":747672540},{"type":"v1","addr":"10.20.193.27:6837","nonce":747672540}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14730,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.27:6839/761293857","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6838","nonce":761293857},{"type":"v1","addr":"10.20.193.27:6839","nonce":761293857}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":14736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.27:6841/1659204922","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6840","nonce":1659204922},{"type":"v1","addr":"10.20.193.27:6841","nonce":1659204922}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.107:6833/3112510413","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6832","nonce":3112510413},{"type":"v1","addr":"10.20.193.107:6833","nonce":3112510413}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6833/2774006818","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6832","nonce":2774006818},{"type":"v1","addr":"10.20.193.28:6833","nonce":2774006818}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.107:6835/3281057176","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6834","nonce":3281057176},{"type":"v1","addr":"10.20.193.107:6835","nonce":3281057176}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.107:6837/2849542469","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6836","nonce":2849542469},{"type":"v1","addr":"10.20.193.107:6837","nonce":2849542469}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/1510901141","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":1510901141},{"type":"v1","addr":"10.20.193.28:6835","nonce":1510901141}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.107:6839/3260223906","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6838","nonce":3260223906},{"type":"v1","addr":"10.20.193.107:6839","nonce":3260223906}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/543345874","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":543345874},{"type":"v1","addr":"10.20.193.28:6837","nonce":543345874}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6839/2398862063","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":2398862063},{"type":"v1","addr":"10.20.193.28:6839","nonce":2398862063}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2026-02-13T02:22:13.516 INFO:teuthology.orchestra.run.trial027.stderr:dumped fsmap epoch 11 2026-02-13T02:22:13.516 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.512+0000 7f2a506896c0 1 -- 10.20.193.27:0/1599540744 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f2a240b5680 msgr2=0x7f2a240d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:13.516 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.512+0000 7f2a506896c0 1 --2- 10.20.193.27:0/1599540744 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f2a240b5680 0x7f2a240d5b30 secure :-1 s=READY pgs=161 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2a38005980 tx=0x7f2a38005910 comp rx=0 tx=0).stop 2026-02-13T02:22:13.516 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.512+0000 7f2a506896c0 1 -- 10.20.193.27:0/1599540744 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2a48127010 msgr2=0x7f2a481274b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:13.516 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.512+0000 7f2a506896c0 1 --2- 10.20.193.27:0/1599540744 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2a48127010 0x7f2a481274b0 secure :-1 s=READY pgs=210 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2a4400a390 tx=0x7f2a44005530 comp rx=0 tx=0).stop 2026-02-13T02:22:13.516 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.512+0000 7f2a506896c0 1 -- 10.20.193.27:0/1599540744 shutdown_connections 2026-02-13T02:22:13.516 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.512+0000 7f2a506896c0 1 --2- 10.20.193.27:0/1599540744 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f2a240b5680 0x7f2a240d5b30 unknown :-1 s=CLOSED pgs=161 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:13.517 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.512+0000 7f2a506896c0 1 --2- 10.20.193.27:0/1599540744 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2a48127010 0x7f2a481274b0 unknown :-1 s=CLOSED pgs=210 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:13.517 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.512+0000 7f2a506896c0 1 --2- 10.20.193.27:0/1599540744 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2a481198f0 0x7f2a48121c50 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:13.517 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.512+0000 7f2a506896c0 1 --2- 10.20.193.27:0/1599540744 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2a481137a0 0x7f2a48121710 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:13.517 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.512+0000 7f2a506896c0 1 -- 10.20.193.27:0/1599540744 >> 10.20.193.27:0/1599540744 conn(0x7f2a48060760 msgr2=0x7f2a48114ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:13.517 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.512+0000 7f2a506896c0 1 -- 10.20.193.27:0/1599540744 shutdown_connections 2026-02-13T02:22:13.517 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.512+0000 7f2a506896c0 1 -- 10.20.193.27:0/1599540744 wait complete. 2026-02-13T02:22:13.558 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph --format=json-pretty osd lspools 2026-02-13T02:22:13.674 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:13.796 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.792+0000 7fe3b44ac6c0 1 -- 10.20.193.27:0/2813969676 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe3ac115440 msgr2=0x7fe3ac175de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:13.797 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.792+0000 7fe3b44ac6c0 1 --2- 10.20.193.27:0/2813969676 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe3ac115440 0x7fe3ac175de0 secure :-1 s=READY pgs=211 gs=1 cs=0 l=1 c_cookie=14618628975349095285 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe3a800aa70 tx=0x7fe3a802fde0 comp rx=0 tx=0).stop 2026-02-13T02:22:13.797 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.792+0000 7fe3b44ac6c0 1 -- 10.20.193.27:0/2813969676 shutdown_connections 2026-02-13T02:22:13.797 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.792+0000 7fe3b44ac6c0 1 --2- 10.20.193.27:0/2813969676 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe3ac115440 0x7fe3ac175de0 unknown :-1 s=CLOSED pgs=211 gs=1 cs=0 l=1 c_cookie=14618628975349095285 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:13.797 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.792+0000 7fe3b44ac6c0 1 --2- 10.20.193.27:0/2813969676 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe3ac10fc20 0x7fe3ac114c60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:13.797 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.792+0000 7fe3b44ac6c0 1 --2- 10.20.193.27:0/2813969676 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe3ac10f230 0x7fe3ac10f600 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:13.797 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.792+0000 7fe3b44ac6c0 1 -- 10.20.193.27:0/2813969676 >> 10.20.193.27:0/2813969676 conn(0x7fe3ac0ecb30 msgr2=0x7fe3ac10cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:13.797 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.792+0000 7fe3b44ac6c0 1 -- 10.20.193.27:0/2813969676 shutdown_connections 2026-02-13T02:22:13.797 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.792+0000 7fe3b44ac6c0 1 -- 10.20.193.27:0/2813969676 wait complete. 2026-02-13T02:22:13.797 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.793+0000 7fe3b44ac6c0 1 Processor -- start 2026-02-13T02:22:13.797 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.793+0000 7fe3b44ac6c0 1 -- start start 2026-02-13T02:22:13.797 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.793+0000 7fe3b44ac6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe3ac10f230 0x7fe3ac11d210 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:13.798 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.793+0000 7fe3b44ac6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe3ac115440 0x7fe3ac11d750 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:13.798 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.793+0000 7fe3b44ac6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe3ac122b10 0x7fe3ac122fb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:13.798 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.793+0000 7fe3b44ac6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ac123520 con 0x7fe3ac115440 2026-02-13T02:22:13.798 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.793+0000 7fe3b44ac6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ac123690 con 0x7fe3ac122b10 2026-02-13T02:22:13.798 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.793+0000 7fe3b44ac6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ac123850 con 0x7fe3ac10f230 2026-02-13T02:22:13.798 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.793+0000 7fe3b1a206c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe3ac115440 0x7fe3ac11d750 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:13.798 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.793+0000 7fe3b1a206c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe3ac115440 0x7fe3ac11d750 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:37620/0 (socket says 10.20.193.27:37620) 2026-02-13T02:22:13.798 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.793+0000 7fe3b1a206c0 1 -- 10.20.193.27:0/2367726889 learned_addr learned my addr 10.20.193.27:0/2367726889 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:13.798 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.793+0000 7fe3b22216c0 1 --2- 10.20.193.27:0/2367726889 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe3ac10f230 0x7fe3ac11d210 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:13.798 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.793+0000 7fe3b2a226c0 1 --2- 10.20.193.27:0/2367726889 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe3ac122b10 0x7fe3ac122fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:13.798 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.793+0000 7fe3b1a206c0 1 -- 10.20.193.27:0/2367726889 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe3ac10f230 msgr2=0x7fe3ac11d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:13.798 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.793+0000 7fe3b1a206c0 1 --2- 10.20.193.27:0/2367726889 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe3ac10f230 0x7fe3ac11d210 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:13.798 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.793+0000 7fe3b1a206c0 1 -- 10.20.193.27:0/2367726889 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe3ac122b10 msgr2=0x7fe3ac122fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:13.798 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.793+0000 7fe3b1a206c0 1 --2- 10.20.193.27:0/2367726889 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe3ac122b10 0x7fe3ac122fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:13.798 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.793+0000 7fe3b1a206c0 1 -- 10.20.193.27:0/2367726889 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a800a660 con 0x7fe3ac115440 2026-02-13T02:22:13.799 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.793+0000 7fe3b2a226c0 1 --2- 10.20.193.27:0/2367726889 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe3ac122b10 0x7fe3ac122fb0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:22:13.799 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.793+0000 7fe3b1a206c0 1 --2- 10.20.193.27:0/2367726889 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe3ac115440 0x7fe3ac11d750 secure :-1 s=READY pgs=212 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe3a0005fd0 tx=0x7fe3a000bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:13.799 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.793+0000 7fe3937fe6c0 1 -- 10.20.193.27:0/2367726889 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe3a0004280 con 0x7fe3ac115440 2026-02-13T02:22:13.799 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.793+0000 7fe3937fe6c0 1 -- 10.20.193.27:0/2367726889 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fe3a00043e0 con 0x7fe3ac115440 2026-02-13T02:22:13.799 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.793+0000 7fe3b44ac6c0 1 -- 10.20.193.27:0/2367726889 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3ac163d00 con 0x7fe3ac115440 2026-02-13T02:22:13.799 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.793+0000 7fe3937fe6c0 1 -- 10.20.193.27:0/2367726889 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe3a000ccd0 con 0x7fe3ac115440 2026-02-13T02:22:13.799 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.793+0000 7fe3b44ac6c0 1 -- 10.20.193.27:0/2367726889 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3ac164190 con 0x7fe3ac115440 2026-02-13T02:22:13.799 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.794+0000 7fe3b44ac6c0 1 -- 10.20.193.27:0/2367726889 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3ac110500 con 0x7fe3ac115440 2026-02-13T02:22:13.800 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.795+0000 7fe3937fe6c0 1 -- 10.20.193.27:0/2367726889 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7fe3a00026b0 con 0x7fe3ac115440 2026-02-13T02:22:13.800 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.796+0000 7fe3937fe6c0 1 --2- 10.20.193.27:0/2367726889 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe3880b5600 0x7fe3880d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:13.800 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.796+0000 7fe3b22216c0 1 --2- 10.20.193.27:0/2367726889 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe3880b5600 0x7fe3880d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:13.800 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.796+0000 7fe3937fe6c0 1 -- 10.20.193.27:0/2367726889 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6904+0+0 (secure 0 0 0) 0x7fe3a00e3710 con 0x7fe3ac115440 2026-02-13T02:22:13.800 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.796+0000 7fe3937fe6c0 1 -- 10.20.193.27:0/2367726889 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fe3a00a65f0 con 0x7fe3ac115440 2026-02-13T02:22:13.800 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.796+0000 7fe3b22216c0 1 --2- 10.20.193.27:0/2367726889 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe3880b5600 0x7fe3880d5ab0 secure :-1 s=READY pgs=162 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe39c006b60 tx=0x7fe39c006af0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:13.893 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.891+0000 7fe3b44ac6c0 1 -- 10.20.193.27:0/2367726889 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe3ac10f600 con 0x7fe3ac115440 2026-02-13T02:22:13.894 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.892+0000 7fe3937fe6c0 1 -- 10.20.193.27:0/2367726889 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v66) ==== 84+0+65 (secure 0 0 0) 0x7fe3a00a97a0 con 0x7fe3ac115440 2026-02-13T02:22:13.894 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:22:13.894 INFO:teuthology.orchestra.run.trial027.stdout:[ 2026-02-13T02:22:13.894 INFO:teuthology.orchestra.run.trial027.stdout: { 2026-02-13T02:22:13.894 INFO:teuthology.orchestra.run.trial027.stdout: "poolnum": 1, 2026-02-13T02:22:13.894 INFO:teuthology.orchestra.run.trial027.stdout: "poolname": ".mgr" 2026-02-13T02:22:13.894 INFO:teuthology.orchestra.run.trial027.stdout: } 2026-02-13T02:22:13.894 INFO:teuthology.orchestra.run.trial027.stdout:] 2026-02-13T02:22:13.895 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.893+0000 7fe3b44ac6c0 1 -- 10.20.193.27:0/2367726889 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe3880b5600 msgr2=0x7fe3880d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:13.896 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.893+0000 7fe3b44ac6c0 1 --2- 10.20.193.27:0/2367726889 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe3880b5600 0x7fe3880d5ab0 secure :-1 s=READY pgs=162 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe39c006b60 tx=0x7fe39c006af0 comp rx=0 tx=0).stop 2026-02-13T02:22:13.896 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.893+0000 7fe3b44ac6c0 1 -- 10.20.193.27:0/2367726889 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe3ac115440 msgr2=0x7fe3ac11d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:13.896 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.893+0000 7fe3b44ac6c0 1 --2- 10.20.193.27:0/2367726889 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe3ac115440 0x7fe3ac11d750 secure :-1 s=READY pgs=212 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe3a0005fd0 tx=0x7fe3a000bca0 comp rx=0 tx=0).stop 2026-02-13T02:22:13.896 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.894+0000 7fe3b44ac6c0 1 -- 10.20.193.27:0/2367726889 shutdown_connections 2026-02-13T02:22:13.896 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.894+0000 7fe3b44ac6c0 1 --2- 10.20.193.27:0/2367726889 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe3880b5600 0x7fe3880d5ab0 unknown :-1 s=CLOSED pgs=162 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:13.896 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.894+0000 7fe3b44ac6c0 1 --2- 10.20.193.27:0/2367726889 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe3ac122b10 0x7fe3ac122fb0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:13.896 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.894+0000 7fe3b44ac6c0 1 --2- 10.20.193.27:0/2367726889 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe3ac115440 0x7fe3ac11d750 unknown :-1 s=CLOSED pgs=212 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:13.896 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.894+0000 7fe3b44ac6c0 1 --2- 10.20.193.27:0/2367726889 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe3ac10f230 0x7fe3ac11d210 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:13.896 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.894+0000 7fe3b44ac6c0 1 -- 10.20.193.27:0/2367726889 >> 10.20.193.27:0/2367726889 conn(0x7fe3ac0ecb30 msgr2=0x7fe3ac10cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:13.896 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.894+0000 7fe3b44ac6c0 1 -- 10.20.193.27:0/2367726889 shutdown_connections 2026-02-13T02:22:13.896 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:13.894+0000 7fe3b44ac6c0 1 -- 10.20.193.27:0/2367726889 wait complete. 2026-02-13T02:22:13.944 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2026-02-13T02:22:13.944 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2026-02-13T02:22:13.993 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:13 trial028 ceph-mon[26701]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:13.993 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:13 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:22:13.993 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:13 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1599540744' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-13T02:22:14.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:13 trial107 ceph-mon[27422]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:14.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:13 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:22:14.000 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:13 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1599540744' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-13T02:22:14.062 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:14.084 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:13 trial027 ceph-mon[24567]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:14.084 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:13 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:22:14.084 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:13 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1599540744' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-13T02:22:14.180 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.177+0000 7f8f725a36c0 1 -- 10.20.193.27:0/3439597342 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8f6c115440 msgr2=0x7f8f6c175de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:14.180 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.177+0000 7f8f725a36c0 1 --2- 10.20.193.27:0/3439597342 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8f6c115440 0x7f8f6c175de0 secure :-1 s=READY pgs=213 gs=1 cs=0 l=1 c_cookie=15071612432257991668 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8f6000aa70 tx=0x7f8f6002fde0 comp rx=0 tx=0).stop 2026-02-13T02:22:14.180 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.177+0000 7f8f725a36c0 1 -- 10.20.193.27:0/3439597342 shutdown_connections 2026-02-13T02:22:14.180 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.177+0000 7f8f725a36c0 1 --2- 10.20.193.27:0/3439597342 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8f6c115440 0x7f8f6c175de0 unknown :-1 s=CLOSED pgs=213 gs=1 cs=0 l=1 c_cookie=15071612432257991668 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:14.180 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.177+0000 7f8f725a36c0 1 --2- 10.20.193.27:0/3439597342 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8f6c10fc20 0x7f8f6c114c60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:14.180 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.177+0000 7f8f725a36c0 1 --2- 10.20.193.27:0/3439597342 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8f6c10f230 0x7f8f6c10f600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:14.180 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.177+0000 7f8f725a36c0 1 -- 10.20.193.27:0/3439597342 >> 10.20.193.27:0/3439597342 conn(0x7f8f6c0ecb30 msgr2=0x7f8f6c10cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:14.181 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.177+0000 7f8f725a36c0 1 -- 10.20.193.27:0/3439597342 shutdown_connections 2026-02-13T02:22:14.181 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.177+0000 7f8f725a36c0 1 -- 10.20.193.27:0/3439597342 wait complete. 2026-02-13T02:22:14.181 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.178+0000 7f8f725a36c0 1 Processor -- start 2026-02-13T02:22:14.181 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.178+0000 7f8f725a36c0 1 -- start start 2026-02-13T02:22:14.181 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.178+0000 7f8f725a36c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8f6c10f230 0x7f8f6c11d280 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:14.181 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.178+0000 7f8f725a36c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8f6c115440 0x7f8f6c11d7c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:14.181 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.178+0000 7f8f725a36c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8f6c122b80 0x7f8f6c123020 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:14.181 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.178+0000 7f8f725a36c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c123590 con 0x7f8f6c122b80 2026-02-13T02:22:14.181 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.178+0000 7f8f725a36c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c123700 con 0x7f8f6c10f230 2026-02-13T02:22:14.181 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.178+0000 7f8f725a36c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c1238c0 con 0x7f8f6c115440 2026-02-13T02:22:14.181 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.178+0000 7f8f70b156c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8f6c122b80 0x7f8f6c123020 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:14.181 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.178+0000 7f8f70b156c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8f6c122b80 0x7f8f6c123020 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:37640/0 (socket says 10.20.193.27:37640) 2026-02-13T02:22:14.181 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.178+0000 7f8f70b156c0 1 -- 10.20.193.27:0/3342072812 learned_addr learned my addr 10.20.193.27:0/3342072812 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:14.181 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.178+0000 7f8f6bfff6c0 1 --2- 10.20.193.27:0/3342072812 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8f6c10f230 0x7f8f6c11d280 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:14.181 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.178+0000 7f8f70b156c0 1 -- 10.20.193.27:0/3342072812 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8f6c115440 msgr2=0x7f8f6c11d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:14.181 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.178+0000 7f8f70b156c0 1 --2- 10.20.193.27:0/3342072812 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8f6c115440 0x7f8f6c11d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:14.181 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.178+0000 7f8f70b156c0 1 -- 10.20.193.27:0/3342072812 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8f6c10f230 msgr2=0x7f8f6c11d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:14.181 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.178+0000 7f8f70b156c0 1 --2- 10.20.193.27:0/3342072812 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8f6c10f230 0x7f8f6c11d280 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:14.181 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.178+0000 7f8f70b156c0 1 -- 10.20.193.27:0/3342072812 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6000a660 con 0x7f8f6c122b80 2026-02-13T02:22:14.181 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.178+0000 7f8f70b156c0 1 --2- 10.20.193.27:0/3342072812 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8f6c122b80 0x7f8f6c123020 secure :-1 s=READY pgs=214 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8f600054b0 tx=0x7f8f600054e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:14.181 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.178+0000 7f8f697fa6c0 1 -- 10.20.193.27:0/3342072812 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8f60003a30 con 0x7f8f6c122b80 2026-02-13T02:22:14.182 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.178+0000 7f8f725a36c0 1 -- 10.20.193.27:0/3342072812 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f6c163d10 con 0x7f8f6c122b80 2026-02-13T02:22:14.182 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.178+0000 7f8f725a36c0 1 -- 10.20.193.27:0/3342072812 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f6c164140 con 0x7f8f6c122b80 2026-02-13T02:22:14.182 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.178+0000 7f8f697fa6c0 1 -- 10.20.193.27:0/3342072812 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f8f60003b90 con 0x7f8f6c122b80 2026-02-13T02:22:14.182 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.178+0000 7f8f697fa6c0 1 -- 10.20.193.27:0/3342072812 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8f600372f0 con 0x7f8f6c122b80 2026-02-13T02:22:14.182 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.179+0000 7f8f725a36c0 1 -- 10.20.193.27:0/3342072812 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f6c110500 con 0x7f8f6c122b80 2026-02-13T02:22:14.182 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.180+0000 7f8f697fa6c0 1 -- 10.20.193.27:0/3342072812 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7f8f60008260 con 0x7f8f6c122b80 2026-02-13T02:22:14.183 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.181+0000 7f8f697fa6c0 1 --2- 10.20.193.27:0/3342072812 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8f400b5680 0x7f8f400d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:14.183 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.181+0000 7f8f6bfff6c0 1 --2- 10.20.193.27:0/3342072812 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8f400b5680 0x7f8f400d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:14.183 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.181+0000 7f8f697fa6c0 1 -- 10.20.193.27:0/3342072812 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6904+0+0 (secure 0 0 0) 0x7f8f60108970 con 0x7f8f6c122b80 2026-02-13T02:22:14.183 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.181+0000 7f8f697fa6c0 1 -- 10.20.193.27:0/3342072812 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f8f60108df0 con 0x7f8f6c122b80 2026-02-13T02:22:14.183 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.181+0000 7f8f6bfff6c0 1 --2- 10.20.193.27:0/3342072812 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8f400b5680 0x7f8f400d5b30 secure :-1 s=READY pgs=163 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8f58004750 tx=0x7f8f58009390 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:14.276 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.274+0000 7f8f725a36c0 1 -- 10.20.193.27:0/3342072812 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8f6c10f600 con 0x7f8f6c122b80 2026-02-13T02:22:14.909 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:14 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2367726889' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2026-02-13T02:22:14.910 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:14.910 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:14.910 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:14.910 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:14.910 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:14.910 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:14.910 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:14 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3342072812' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2026-02-13T02:22:14.910 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:14.910 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:22:14.911 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:14.911 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:14 trial107 ceph-mon[27422]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:14.911 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:22:14.911 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:22:14.911 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:14 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:15.002 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:14.999+0000 7f8f697fa6c0 1 -- 10.20.193.27:0/3342072812 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v67) ==== 138+0+0 (secure 0 0 0) 0x7f8f600cb850 con 0x7f8f6c122b80 2026-02-13T02:22:15.002 INFO:teuthology.orchestra.run.trial027.stderr:pool 'cephfs_metadata' created 2026-02-13T02:22:15.005 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.002+0000 7f8f725a36c0 1 -- 10.20.193.27:0/3342072812 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8f400b5680 msgr2=0x7f8f400d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:15.005 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.002+0000 7f8f725a36c0 1 --2- 10.20.193.27:0/3342072812 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8f400b5680 0x7f8f400d5b30 secure :-1 s=READY pgs=163 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8f58004750 tx=0x7f8f58009390 comp rx=0 tx=0).stop 2026-02-13T02:22:15.005 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.002+0000 7f8f725a36c0 1 -- 10.20.193.27:0/3342072812 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8f6c122b80 msgr2=0x7f8f6c123020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:15.005 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.002+0000 7f8f725a36c0 1 --2- 10.20.193.27:0/3342072812 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8f6c122b80 0x7f8f6c123020 secure :-1 s=READY pgs=214 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8f600054b0 tx=0x7f8f600054e0 comp rx=0 tx=0).stop 2026-02-13T02:22:15.005 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.002+0000 7f8f725a36c0 1 -- 10.20.193.27:0/3342072812 shutdown_connections 2026-02-13T02:22:15.006 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.002+0000 7f8f725a36c0 1 --2- 10.20.193.27:0/3342072812 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8f400b5680 0x7f8f400d5b30 unknown :-1 s=CLOSED pgs=163 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:15.006 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.002+0000 7f8f725a36c0 1 --2- 10.20.193.27:0/3342072812 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8f6c122b80 0x7f8f6c123020 unknown :-1 s=CLOSED pgs=214 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:15.006 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.002+0000 7f8f725a36c0 1 --2- 10.20.193.27:0/3342072812 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8f6c115440 0x7f8f6c11d7c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:15.006 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.002+0000 7f8f725a36c0 1 --2- 10.20.193.27:0/3342072812 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8f6c10f230 0x7f8f6c11d280 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:15.006 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.002+0000 7f8f725a36c0 1 -- 10.20.193.27:0/3342072812 >> 10.20.193.27:0/3342072812 conn(0x7f8f6c0ecb30 msgr2=0x7f8f6c10cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:15.006 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.002+0000 7f8f725a36c0 1 -- 10.20.193.27:0/3342072812 shutdown_connections 2026-02-13T02:22:15.006 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.002+0000 7f8f725a36c0 1 -- 10.20.193.27:0/3342072812 wait complete. 2026-02-13T02:22:15.046 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:14 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2367726889' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2026-02-13T02:22:15.046 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:15.046 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:15.046 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:15.046 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:15.046 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:15.046 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:15.046 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:14 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3342072812' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2026-02-13T02:22:15.046 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:15.047 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:22:15.047 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:15.047 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:14 trial027 ceph-mon[24567]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:15.047 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:22:15.047 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:22:15.047 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:14 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:15.048 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2026-02-13T02:22:15.164 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:15.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:14 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2367726889' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2026-02-13T02:22:15.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:15.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:15.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:15.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:15.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:15.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:15.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:14 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3342072812' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2026-02-13T02:22:15.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:15.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:22:15.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:22:15.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:14 trial028 ceph-mon[26701]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:15.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:22:15.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:22:15.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:14 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:22:15.281 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.278+0000 7f11626676c0 1 -- 10.20.193.27:0/3421427085 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f115c10d9a0 msgr2=0x7f115c1129e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:15.281 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.278+0000 7f11626676c0 1 --2- 10.20.193.27:0/3421427085 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f115c10d9a0 0x7f115c1129e0 secure :-1 s=READY pgs=215 gs=1 cs=0 l=1 c_cookie=14070246373513093747 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f114c0099a0 tx=0x7f114c02ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:15.281 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.278+0000 7f11626676c0 1 -- 10.20.193.27:0/3421427085 shutdown_connections 2026-02-13T02:22:15.281 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.278+0000 7f11626676c0 1 --2- 10.20.193.27:0/3421427085 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f115c1131c0 0x7f115c173b60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:15.281 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.278+0000 7f11626676c0 1 --2- 10.20.193.27:0/3421427085 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f115c10d9a0 0x7f115c1129e0 unknown :-1 s=CLOSED pgs=215 gs=1 cs=0 l=1 c_cookie=14070246373513093747 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:15.281 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.278+0000 7f11626676c0 1 --2- 10.20.193.27:0/3421427085 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f115c10cfb0 0x7f115c10d380 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:15.281 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.278+0000 7f11626676c0 1 -- 10.20.193.27:0/3421427085 >> 10.20.193.27:0/3421427085 conn(0x7f115c0ea8b0 msgr2=0x7f115c10acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:15.281 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.278+0000 7f11626676c0 1 -- 10.20.193.27:0/3421427085 shutdown_connections 2026-02-13T02:22:15.281 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.278+0000 7f11626676c0 1 -- 10.20.193.27:0/3421427085 wait complete. 2026-02-13T02:22:15.281 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.279+0000 7f11626676c0 1 Processor -- start 2026-02-13T02:22:15.281 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.279+0000 7f11626676c0 1 -- start start 2026-02-13T02:22:15.282 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.279+0000 7f11626676c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f115c10cfb0 0x7f115c0f2ae0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:15.282 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.279+0000 7f11626676c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f115c1131c0 0x7f115c173a60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:15.282 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.279+0000 7f11626676c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f115c0f3110 0x7f115c0f8390 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:15.282 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.279+0000 7f11626676c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f115c0f3800 con 0x7f115c10cfb0 2026-02-13T02:22:15.282 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.279+0000 7f11626676c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f115c0f88d0 con 0x7f115c0f3110 2026-02-13T02:22:15.282 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.279+0000 7f11626676c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f115c0f8a90 con 0x7f115c1131c0 2026-02-13T02:22:15.282 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.279+0000 7f115bfff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f115c10cfb0 0x7f115c0f2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:15.282 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.279+0000 7f115bfff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f115c10cfb0 0x7f115c0f2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:37658/0 (socket says 10.20.193.27:37658) 2026-02-13T02:22:15.282 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.279+0000 7f115bfff6c0 1 -- 10.20.193.27:0/1897077955 learned_addr learned my addr 10.20.193.27:0/1897077955 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:15.282 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.279+0000 7f115b7fe6c0 1 --2- 10.20.193.27:0/1897077955 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f115c1131c0 0x7f115c173a60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:15.282 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.279+0000 7f115bfff6c0 1 -- 10.20.193.27:0/1897077955 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f115c1131c0 msgr2=0x7f115c173a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:15.282 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.279+0000 7f115bfff6c0 1 --2- 10.20.193.27:0/1897077955 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f115c1131c0 0x7f115c173a60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:15.283 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.279+0000 7f115bfff6c0 1 -- 10.20.193.27:0/1897077955 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f115c0f3110 msgr2=0x7f115c0f8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:15.283 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.279+0000 7f115bfff6c0 1 --2- 10.20.193.27:0/1897077955 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f115c0f3110 0x7f115c0f8390 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:15.283 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.279+0000 7f115bfff6c0 1 -- 10.20.193.27:0/1897077955 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114c009590 con 0x7f115c10cfb0 2026-02-13T02:22:15.283 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.279+0000 7f115bfff6c0 1 --2- 10.20.193.27:0/1897077955 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f115c10cfb0 0x7f115c0f2ae0 secure :-1 s=READY pgs=216 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1148005dc0 tx=0x7f114800bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:15.283 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.279+0000 7f11597fa6c0 1 -- 10.20.193.27:0/1897077955 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1148004250 con 0x7f115c10cfb0 2026-02-13T02:22:15.283 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.279+0000 7f11626676c0 1 -- 10.20.193.27:0/1897077955 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f115c0f8d70 con 0x7f115c10cfb0 2026-02-13T02:22:15.283 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.279+0000 7f11597fa6c0 1 -- 10.20.193.27:0/1897077955 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1148009070 con 0x7f115c10cfb0 2026-02-13T02:22:15.283 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.279+0000 7f11597fa6c0 1 -- 10.20.193.27:0/1897077955 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f114800cde0 con 0x7f115c10cfb0 2026-02-13T02:22:15.283 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.279+0000 7f11626676c0 1 -- 10.20.193.27:0/1897077955 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f115c0f9200 con 0x7f115c10cfb0 2026-02-13T02:22:15.283 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.280+0000 7f11626676c0 1 -- 10.20.193.27:0/1897077955 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f115c10e280 con 0x7f115c10cfb0 2026-02-13T02:22:15.284 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.282+0000 7f11597fa6c0 1 -- 10.20.193.27:0/1897077955 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7f11480028c0 con 0x7f115c10cfb0 2026-02-13T02:22:15.284 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.282+0000 7f11597fa6c0 1 --2- 10.20.193.27:0/1897077955 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f11300b5600 0x7f11300d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:15.284 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.282+0000 7f115b7fe6c0 1 --2- 10.20.193.27:0/1897077955 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f11300b5600 0x7f11300d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:15.285 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.283+0000 7f11597fa6c0 1 -- 10.20.193.27:0/1897077955 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 7277+0+0 (secure 0 0 0) 0x7f11480e3590 con 0x7f115c10cfb0 2026-02-13T02:22:15.285 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.283+0000 7f115b7fe6c0 1 --2- 10.20.193.27:0/1897077955 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f11300b5600 0x7f11300d5ab0 secure :-1 s=READY pgs=164 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f114c005b40 tx=0x7f114c0092a0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:15.285 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.283+0000 7f11597fa6c0 1 -- 10.20.193.27:0/1897077955 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f11480a6340 con 0x7f115c10cfb0 2026-02-13T02:22:15.379 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:15.376+0000 7f11626676c0 1 -- 10.20.193.27:0/1897077955 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f115c0f9200 con 0x7f115c10cfb0 2026-02-13T02:22:16.004 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.001+0000 7f11597fa6c0 1 -- 10.20.193.27:0/1897077955 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v68) ==== 170+0+0 (secure 0 0 0) 0x7f115c0f9200 con 0x7f115c10cfb0 2026-02-13T02:22:16.004 INFO:teuthology.orchestra.run.trial027.stderr:pool 'cephfs_data' created 2026-02-13T02:22:16.008 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.005+0000 7f11626676c0 1 -- 10.20.193.27:0/1897077955 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f11300b5600 msgr2=0x7f11300d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:16.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.005+0000 7f11626676c0 1 --2- 10.20.193.27:0/1897077955 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f11300b5600 0x7f11300d5ab0 secure :-1 s=READY pgs=164 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f114c005b40 tx=0x7f114c0092a0 comp rx=0 tx=0).stop 2026-02-13T02:22:16.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.005+0000 7f11626676c0 1 -- 10.20.193.27:0/1897077955 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f115c10cfb0 msgr2=0x7f115c0f2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:16.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.005+0000 7f11626676c0 1 --2- 10.20.193.27:0/1897077955 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f115c10cfb0 0x7f115c0f2ae0 secure :-1 s=READY pgs=216 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1148005dc0 tx=0x7f114800bbb0 comp rx=0 tx=0).stop 2026-02-13T02:22:16.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.006+0000 7f11626676c0 1 -- 10.20.193.27:0/1897077955 shutdown_connections 2026-02-13T02:22:16.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.006+0000 7f11626676c0 1 --2- 10.20.193.27:0/1897077955 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f11300b5600 0x7f11300d5ab0 unknown :-1 s=CLOSED pgs=164 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:16.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.006+0000 7f11626676c0 1 --2- 10.20.193.27:0/1897077955 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f115c0f3110 0x7f115c0f8390 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:16.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.006+0000 7f11626676c0 1 --2- 10.20.193.27:0/1897077955 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f115c1131c0 0x7f115c173a60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:16.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.006+0000 7f11626676c0 1 --2- 10.20.193.27:0/1897077955 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f115c10cfb0 0x7f115c0f2ae0 unknown :-1 s=CLOSED pgs=216 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:16.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.006+0000 7f11626676c0 1 -- 10.20.193.27:0/1897077955 >> 10.20.193.27:0/1897077955 conn(0x7f115c0ea8b0 msgr2=0x7f115c10b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:16.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.006+0000 7f11626676c0 1 -- 10.20.193.27:0/1897077955 shutdown_connections 2026-02-13T02:22:16.010 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.006+0000 7f11626676c0 1 -- 10.20.193.27:0/1897077955 wait complete. 2026-02-13T02:22:16.050 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph fs new cephfs cephfs_metadata cephfs_data 2026-02-13T02:22:16.167 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:16.282 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:16 trial027 ceph-mon[24567]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:16.282 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:16 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3342072812' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2026-02-13T02:22:16.283 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:16 trial027 ceph-mon[24567]: osdmap e67: 12 total, 12 up, 12 in 2026-02-13T02:22:16.283 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:16 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1897077955' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2026-02-13T02:22:16.288 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.284+0000 7fc3ff3186c0 1 -- 10.20.193.27:0/2778853062 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc3f810fbf0 msgr2=0x7fc3f8114c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:16.288 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.284+0000 7fc3ff3186c0 1 --2- 10.20.193.27:0/2778853062 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc3f810fbf0 0x7fc3f8114c30 secure :-1 s=READY pgs=217 gs=1 cs=0 l=1 c_cookie=10438494979935916589 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc3ec0099a0 tx=0x7fc3ec02ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:16.288 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.284+0000 7fc3ff3186c0 1 -- 10.20.193.27:0/2778853062 shutdown_connections 2026-02-13T02:22:16.289 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.284+0000 7fc3ff3186c0 1 --2- 10.20.193.27:0/2778853062 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc3f8115410 0x7fc3f8175db0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:16.289 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.284+0000 7fc3ff3186c0 1 --2- 10.20.193.27:0/2778853062 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc3f810fbf0 0x7fc3f8114c30 unknown :-1 s=CLOSED pgs=217 gs=1 cs=0 l=1 c_cookie=10438494979935916589 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:16.289 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.284+0000 7fc3ff3186c0 1 --2- 10.20.193.27:0/2778853062 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc3f810f200 0x7fc3f810f5d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:16.289 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.284+0000 7fc3ff3186c0 1 -- 10.20.193.27:0/2778853062 >> 10.20.193.27:0/2778853062 conn(0x7fc3f80ecb00 msgr2=0x7fc3f810cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:16.289 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.284+0000 7fc3ff3186c0 1 -- 10.20.193.27:0/2778853062 shutdown_connections 2026-02-13T02:22:16.289 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.284+0000 7fc3ff3186c0 1 -- 10.20.193.27:0/2778853062 wait complete. 2026-02-13T02:22:16.289 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.284+0000 7fc3ff3186c0 1 Processor -- start 2026-02-13T02:22:16.289 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.284+0000 7fc3ff3186c0 1 -- start start 2026-02-13T02:22:16.289 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.284+0000 7fc3ff3186c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc3f810f200 0x7fc3f80f4ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:16.289 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.284+0000 7fc3ff3186c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc3f8115410 0x7fc3f8175cb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:16.290 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.284+0000 7fc3ff3186c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc3f80f5310 0x7fc3f80fa5e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:16.290 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.284+0000 7fc3ff3186c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f80f5a00 con 0x7fc3f80f5310 2026-02-13T02:22:16.290 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.284+0000 7fc3ff3186c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f80fab20 con 0x7fc3f810f200 2026-02-13T02:22:16.290 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.284+0000 7fc3ff3186c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f80face0 con 0x7fc3f8115410 2026-02-13T02:22:16.290 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.284+0000 7fc3fd88a6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc3f80f5310 0x7fc3f80fa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:16.290 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.284+0000 7fc3fd88a6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc3f80f5310 0x7fc3f80fa5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:37680/0 (socket says 10.20.193.27:37680) 2026-02-13T02:22:16.290 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.284+0000 7fc3fd88a6c0 1 -- 10.20.193.27:0/1479855520 learned_addr learned my addr 10.20.193.27:0/1479855520 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:16.290 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.284+0000 7fc3fd0896c0 1 --2- 10.20.193.27:0/1479855520 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc3f810f200 0x7fc3f80f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:16.290 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.284+0000 7fc3fc8886c0 1 --2- 10.20.193.27:0/1479855520 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc3f8115410 0x7fc3f8175cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:16.290 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.284+0000 7fc3fd88a6c0 1 -- 10.20.193.27:0/1479855520 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc3f8115410 msgr2=0x7fc3f8175cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:16.290 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.284+0000 7fc3fd88a6c0 1 --2- 10.20.193.27:0/1479855520 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc3f8115410 0x7fc3f8175cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:16.290 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.284+0000 7fc3fd88a6c0 1 -- 10.20.193.27:0/1479855520 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc3f810f200 msgr2=0x7fc3f80f4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:16.291 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.284+0000 7fc3fd88a6c0 1 --2- 10.20.193.27:0/1479855520 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc3f810f200 0x7fc3f80f4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:16.291 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.284+0000 7fc3fd88a6c0 1 -- 10.20.193.27:0/1479855520 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ec009590 con 0x7fc3f80f5310 2026-02-13T02:22:16.291 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.285+0000 7fc3fd88a6c0 1 --2- 10.20.193.27:0/1479855520 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc3f80f5310 0x7fc3f80fa5e0 secure :-1 s=READY pgs=218 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc3f4006ea0 tx=0x7fc3f400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:16.291 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.285+0000 7fc3e67fc6c0 1 -- 10.20.193.27:0/1479855520 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc3f4003a30 con 0x7fc3f80f5310 2026-02-13T02:22:16.291 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.285+0000 7fc3e67fc6c0 1 -- 10.20.193.27:0/1479855520 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc3f40054f0 con 0x7fc3f80f5310 2026-02-13T02:22:16.291 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.285+0000 7fc3ff3186c0 1 -- 10.20.193.27:0/1479855520 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f80fafc0 con 0x7fc3f80f5310 2026-02-13T02:22:16.291 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.285+0000 7fc3e67fc6c0 1 -- 10.20.193.27:0/1479855520 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc3f400dcd0 con 0x7fc3f80f5310 2026-02-13T02:22:16.291 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.285+0000 7fc3ff3186c0 1 -- 10.20.193.27:0/1479855520 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f80fb750 con 0x7fc3f80f5310 2026-02-13T02:22:16.291 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.285+0000 7fc3ff3186c0 1 -- 10.20.193.27:0/1479855520 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3f80fb4a0 con 0x7fc3f80f5310 2026-02-13T02:22:16.292 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.287+0000 7fc3e67fc6c0 1 -- 10.20.193.27:0/1479855520 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7fc3f4003b90 con 0x7fc3f80f5310 2026-02-13T02:22:16.293 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.287+0000 7fc3e67fc6c0 1 --2- 10.20.193.27:0/1479855520 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc3d40b5600 0x7fc3d40d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:16.293 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.287+0000 7fc3fd0896c0 1 --2- 10.20.193.27:0/1479855520 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc3d40b5600 0x7fc3d40d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:16.293 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.287+0000 7fc3e67fc6c0 1 -- 10.20.193.27:0/1479855520 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7662+0+0 (secure 0 0 0) 0x7fc3f40e4950 con 0x7fc3f80f5310 2026-02-13T02:22:16.293 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.288+0000 7fc3fd0896c0 1 --2- 10.20.193.27:0/1479855520 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc3d40b5600 0x7fc3d40d5ab0 secure :-1 s=READY pgs=165 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc3e8005980 tx=0x7fc3e8005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:16.293 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.288+0000 7fc3e67fc6c0 1 -- 10.20.193.27:0/1479855520 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fc3f40a7580 con 0x7fc3f80f5310 2026-02-13T02:22:16.406 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:16.404+0000 7fc3ff3186c0 1 -- 10.20.193.27:0/1479855520 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc3f81104d0 con 0x7fc3f80f5310 2026-02-13T02:22:16.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:16 trial028 ceph-mon[26701]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:16.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:16 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3342072812' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2026-02-13T02:22:16.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:16 trial028 ceph-mon[26701]: osdmap e67: 12 total, 12 up, 12 in 2026-02-13T02:22:16.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:16 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1897077955' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2026-02-13T02:22:16.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:16 trial107 ceph-mon[27422]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:16.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:16 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3342072812' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2026-02-13T02:22:16.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:16 trial107 ceph-mon[27422]: osdmap e67: 12 total, 12 up, 12 in 2026-02-13T02:22:16.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:16 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1897077955' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2026-02-13T02:22:17.016 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.013+0000 7fc3e67fc6c0 1 -- 10.20.193.27:0/1479855520 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v12) ==== 172+0+0 (secure 0 0 0) 0x7fc3f400de60 con 0x7fc3f80f5310 2026-02-13T02:22:17.016 INFO:teuthology.orchestra.run.trial027.stderr:new fs with metadata pool 2 and data pool 3 2026-02-13T02:22:17.019 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.016+0000 7fc3ff3186c0 1 -- 10.20.193.27:0/1479855520 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc3d40b5600 msgr2=0x7fc3d40d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:17.019 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.016+0000 7fc3ff3186c0 1 --2- 10.20.193.27:0/1479855520 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc3d40b5600 0x7fc3d40d5ab0 secure :-1 s=READY pgs=165 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc3e8005980 tx=0x7fc3e8005910 comp rx=0 tx=0).stop 2026-02-13T02:22:17.019 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.016+0000 7fc3ff3186c0 1 -- 10.20.193.27:0/1479855520 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc3f80f5310 msgr2=0x7fc3f80fa5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:17.019 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.016+0000 7fc3ff3186c0 1 --2- 10.20.193.27:0/1479855520 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc3f80f5310 0x7fc3f80fa5e0 secure :-1 s=READY pgs=218 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc3f4006ea0 tx=0x7fc3f400cca0 comp rx=0 tx=0).stop 2026-02-13T02:22:17.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.017+0000 7fc3ff3186c0 1 -- 10.20.193.27:0/1479855520 shutdown_connections 2026-02-13T02:22:17.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.017+0000 7fc3ff3186c0 1 --2- 10.20.193.27:0/1479855520 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc3d40b5600 0x7fc3d40d5ab0 unknown :-1 s=CLOSED pgs=165 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:17.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.017+0000 7fc3ff3186c0 1 --2- 10.20.193.27:0/1479855520 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc3f80f5310 0x7fc3f80fa5e0 unknown :-1 s=CLOSED pgs=218 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:17.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.017+0000 7fc3ff3186c0 1 --2- 10.20.193.27:0/1479855520 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc3f8115410 0x7fc3f8175cb0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:17.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.017+0000 7fc3ff3186c0 1 --2- 10.20.193.27:0/1479855520 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc3f810f200 0x7fc3f80f4ce0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:17.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.017+0000 7fc3ff3186c0 1 -- 10.20.193.27:0/1479855520 >> 10.20.193.27:0/1479855520 conn(0x7fc3f80ecb00 msgr2=0x7fc3f810d900 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:17.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.017+0000 7fc3ff3186c0 1 -- 10.20.193.27:0/1479855520 shutdown_connections 2026-02-13T02:22:17.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.017+0000 7fc3ff3186c0 1 -- 10.20.193.27:0/1479855520 wait complete. 2026-02-13T02:22:17.062 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd dump --format=json 2026-02-13T02:22:17.179 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:17.293 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:17 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1897077955' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2026-02-13T02:22:17.294 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:17 trial027 ceph-mon[24567]: osdmap e68: 12 total, 12 up, 12 in 2026-02-13T02:22:17.294 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:17 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1479855520' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2026-02-13T02:22:17.294 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:17 trial027 ceph-mon[24567]: daemon mds.b assigned to filesystem cephfs as rank 0 2026-02-13T02:22:17.300 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.296+0000 7f84e96ac6c0 1 -- 10.20.193.27:0/360818531 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f84e410f230 msgr2=0x7f84e410f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:17.301 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.296+0000 7f84e96ac6c0 1 --2- 10.20.193.27:0/360818531 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f84e410f230 0x7f84e410f600 secure :-1 s=READY pgs=219 gs=1 cs=0 l=1 c_cookie=11069358390478218740 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f84d80099a0 tx=0x7f84d802ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:17.301 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.296+0000 7f84e96ac6c0 1 -- 10.20.193.27:0/360818531 shutdown_connections 2026-02-13T02:22:17.301 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.296+0000 7f84e96ac6c0 1 --2- 10.20.193.27:0/360818531 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f84e4115440 0x7f84e4175de0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:17.301 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.296+0000 7f84e96ac6c0 1 --2- 10.20.193.27:0/360818531 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f84e410fc20 0x7f84e4114c60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:17.301 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.296+0000 7f84e96ac6c0 1 --2- 10.20.193.27:0/360818531 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f84e410f230 0x7f84e410f600 unknown :-1 s=CLOSED pgs=219 gs=1 cs=0 l=1 c_cookie=11069358390478218740 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:17.301 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.296+0000 7f84e96ac6c0 1 -- 10.20.193.27:0/360818531 >> 10.20.193.27:0/360818531 conn(0x7f84e40ecb30 msgr2=0x7f84e410cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:17.301 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.296+0000 7f84e96ac6c0 1 -- 10.20.193.27:0/360818531 shutdown_connections 2026-02-13T02:22:17.301 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.296+0000 7f84e96ac6c0 1 -- 10.20.193.27:0/360818531 wait complete. 2026-02-13T02:22:17.302 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.296+0000 7f84e96ac6c0 1 Processor -- start 2026-02-13T02:22:17.302 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.296+0000 7f84e96ac6c0 1 -- start start 2026-02-13T02:22:17.302 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.296+0000 7f84e96ac6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f84e410f230 0x7f84e411d320 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:17.302 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.296+0000 7f84e96ac6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f84e4115440 0x7f84e411d860 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:17.302 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.296+0000 7f84e96ac6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f84e4122c20 0x7f84e41230c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:17.302 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.296+0000 7f84e96ac6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f84e4123630 con 0x7f84e410f230 2026-02-13T02:22:17.302 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.296+0000 7f84e96ac6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f84e41237a0 con 0x7f84e4115440 2026-02-13T02:22:17.302 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.296+0000 7f84e96ac6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f84e4123960 con 0x7f84e4122c20 2026-02-13T02:22:17.302 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.297+0000 7f84e2ffd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f84e410f230 0x7f84e411d320 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:17.302 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.297+0000 7f84e2ffd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f84e410f230 0x7f84e411d320 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55152/0 (socket says 10.20.193.27:55152) 2026-02-13T02:22:17.302 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.297+0000 7f84e2ffd6c0 1 -- 10.20.193.27:0/653159202 learned_addr learned my addr 10.20.193.27:0/653159202 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:17.302 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.297+0000 7f84e27fc6c0 1 --2- 10.20.193.27:0/653159202 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f84e4115440 0x7f84e411d860 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:17.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.297+0000 7f84e37fe6c0 1 --2- 10.20.193.27:0/653159202 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f84e4122c20 0x7f84e41230c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:17.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.297+0000 7f84e2ffd6c0 1 -- 10.20.193.27:0/653159202 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f84e4122c20 msgr2=0x7f84e41230c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:17.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.297+0000 7f84e2ffd6c0 1 --2- 10.20.193.27:0/653159202 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f84e4122c20 0x7f84e41230c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:17.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.297+0000 7f84e2ffd6c0 1 -- 10.20.193.27:0/653159202 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f84e4115440 msgr2=0x7f84e411d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:17.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.297+0000 7f84e2ffd6c0 1 --2- 10.20.193.27:0/653159202 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f84e4115440 0x7f84e411d860 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:17.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.297+0000 7f84e2ffd6c0 1 -- 10.20.193.27:0/653159202 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d8009590 con 0x7f84e410f230 2026-02-13T02:22:17.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.297+0000 7f84e37fe6c0 1 --2- 10.20.193.27:0/653159202 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f84e4122c20 0x7f84e41230c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:22:17.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.297+0000 7f84e2ffd6c0 1 --2- 10.20.193.27:0/653159202 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f84e410f230 0x7f84e411d320 secure :-1 s=READY pgs=220 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f84d802f230 tx=0x7f84d802fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:17.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.297+0000 7f84c3fff6c0 1 -- 10.20.193.27:0/653159202 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f84d8002ba0 con 0x7f84e410f230 2026-02-13T02:22:17.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.297+0000 7f84c3fff6c0 1 -- 10.20.193.27:0/653159202 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f84d8002d00 con 0x7f84e410f230 2026-02-13T02:22:17.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.297+0000 7f84c3fff6c0 1 -- 10.20.193.27:0/653159202 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f84d80372f0 con 0x7f84e410f230 2026-02-13T02:22:17.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.297+0000 7f84e96ac6c0 1 -- 10.20.193.27:0/653159202 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e4163db0 con 0x7f84e410f230 2026-02-13T02:22:17.304 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.297+0000 7f84e96ac6c0 1 -- 10.20.193.27:0/653159202 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e4164230 con 0x7f84e410f230 2026-02-13T02:22:17.304 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.297+0000 7f84e96ac6c0 1 -- 10.20.193.27:0/653159202 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84e4110500 con 0x7f84e410f230 2026-02-13T02:22:17.305 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.299+0000 7f84c3fff6c0 1 -- 10.20.193.27:0/653159202 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7f84d80078d0 con 0x7f84e410f230 2026-02-13T02:22:17.305 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.299+0000 7f84c3fff6c0 1 --2- 10.20.193.27:0/653159202 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f84b80b5680 0x7f84b80d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:17.305 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.299+0000 7f84c3fff6c0 1 -- 10.20.193.27:0/653159202 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7762+0+0 (secure 0 0 0) 0x7f84d8107a70 con 0x7f84e410f230 2026-02-13T02:22:17.305 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.299+0000 7f84e27fc6c0 1 --2- 10.20.193.27:0/653159202 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f84b80b5680 0x7f84b80d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:17.305 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.300+0000 7f84e27fc6c0 1 --2- 10.20.193.27:0/653159202 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f84b80b5680 0x7f84b80d5b30 secure :-1 s=READY pgs=166 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f84cc005980 tx=0x7f84cc005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:17.305 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.300+0000 7f84c3fff6c0 1 -- 10.20.193.27:0/653159202 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f84d80ca630 con 0x7f84e410f230 2026-02-13T02:22:17.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.393+0000 7f84e96ac6c0 1 -- 10.20.193.27:0/653159202 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f84e410f600 con 0x7f84e410f230 2026-02-13T02:22:17.396 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.394+0000 7f84c3fff6c0 1 -- 10.20.193.27:0/653159202 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+22015 (secure 0 0 0) 0x7f84d80cd7e0 con 0x7f84e410f230 2026-02-13T02:22:17.397 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:22:17.398 INFO:teuthology.orchestra.run.trial027.stdout:{"epoch":69,"fsid":"707dbb29-0882-11f1-9262-d404e6e7d460","created":"2026-02-13T02:19:50.420575+0000","modified":"2026-02-13T02:22:16.406005+0000","last_up_change":"2026-02-13T02:21:58.960121+0000","last_in_change":"2026-02-13T02:21:50.695981+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-13T02:20:54.702258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2026-02-13T02:22:14.280021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2026-02-13T02:22:15.383271+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1a7af29f-2999-4be3-a38e-3fc7feb9f2f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":7,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6802","nonce":832083434},{"type":"v1","addr":"10.20.193.27:6803","nonce":832083434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6804","nonce":832083434},{"type":"v1","addr":"10.20.193.27:6805","nonce":832083434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6808","nonce":832083434},{"type":"v1","addr":"10.20.193.27:6809","nonce":832083434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6806","nonce":832083434},{"type":"v1","addr":"10.20.193.27:6807","nonce":832083434}]},"public_addr":"10.20.193.27:6803/832083434","cluster_addr":"10.20.193.27:6805/832083434","heartbeat_back_addr":"10.20.193.27:6809/832083434","heartbeat_front_addr":"10.20.193.27:6807/832083434","state":["exists","up"]},{"osd":1,"uuid":"8c0ad8cf-4150-43f6-8a9c-258b0ebf6a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6800","nonce":4070516162},{"type":"v1","addr":"10.20.193.28:6801","nonce":4070516162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6802","nonce":4070516162},{"type":"v1","addr":"10.20.193.28:6803","nonce":4070516162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6806","nonce":4070516162},{"type":"v1","addr":"10.20.193.28:6807","nonce":4070516162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6804","nonce":4070516162},{"type":"v1","addr":"10.20.193.28:6805","nonce":4070516162}]},"public_addr":"10.20.193.28:6801/4070516162","cluster_addr":"10.20.193.28:6803/4070516162","heartbeat_back_addr":"10.20.193.28:6807/4070516162","heartbeat_front_addr":"10.20.193.28:6805/4070516162","state":["exists","up"]},{"osd":2,"uuid":"a7813078-94d6-4559-a89a-5525161ab5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6800","nonce":4075625103},{"type":"v1","addr":"10.20.193.107:6801","nonce":4075625103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6802","nonce":4075625103},{"type":"v1","addr":"10.20.193.107:6803","nonce":4075625103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6806","nonce":4075625103},{"type":"v1","addr":"10.20.193.107:6807","nonce":4075625103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6804","nonce":4075625103},{"type":"v1","addr":"10.20.193.107:6805","nonce":4075625103}]},"public_addr":"10.20.193.107:6801/4075625103","cluster_addr":"10.20.193.107:6803/4075625103","heartbeat_back_addr":"10.20.193.107:6807/4075625103","heartbeat_front_addr":"10.20.193.107:6805/4075625103","state":["exists","up"]},{"osd":3,"uuid":"268f9714-faed-452b-ae34-3240fa23153a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6810","nonce":382071678},{"type":"v1","addr":"10.20.193.27:6811","nonce":382071678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6812","nonce":382071678},{"type":"v1","addr":"10.20.193.27:6813","nonce":382071678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6816","nonce":382071678},{"type":"v1","addr":"10.20.193.27:6817","nonce":382071678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6814","nonce":382071678},{"type":"v1","addr":"10.20.193.27:6815","nonce":382071678}]},"public_addr":"10.20.193.27:6811/382071678","cluster_addr":"10.20.193.27:6813/382071678","heartbeat_back_addr":"10.20.193.27:6817/382071678","heartbeat_front_addr":"10.20.193.27:6815/382071678","state":["exists","up"]},{"osd":4,"uuid":"4beaec0d-7b39-4bb0-9770-3dc4c848b106","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6808","nonce":711804032},{"type":"v1","addr":"10.20.193.28:6809","nonce":711804032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6810","nonce":711804032},{"type":"v1","addr":"10.20.193.28:6811","nonce":711804032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6814","nonce":711804032},{"type":"v1","addr":"10.20.193.28:6815","nonce":711804032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6812","nonce":711804032},{"type":"v1","addr":"10.20.193.28:6813","nonce":711804032}]},"public_addr":"10.20.193.28:6809/711804032","cluster_addr":"10.20.193.28:6811/711804032","heartbeat_back_addr":"10.20.193.28:6815/711804032","heartbeat_front_addr":"10.20.193.28:6813/711804032","state":["exists","up"]},{"osd":5,"uuid":"bb16754e-66f2-4951-9f31-d1dde591ec92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6808","nonce":4246100128},{"type":"v1","addr":"10.20.193.107:6809","nonce":4246100128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6810","nonce":4246100128},{"type":"v1","addr":"10.20.193.107:6811","nonce":4246100128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6814","nonce":4246100128},{"type":"v1","addr":"10.20.193.107:6815","nonce":4246100128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6812","nonce":4246100128},{"type":"v1","addr":"10.20.193.107:6813","nonce":4246100128}]},"public_addr":"10.20.193.107:6809/4246100128","cluster_addr":"10.20.193.107:6811/4246100128","heartbeat_back_addr":"10.20.193.107:6815/4246100128","heartbeat_front_addr":"10.20.193.107:6813/4246100128","state":["exists","up"]},{"osd":6,"uuid":"318afff7-b7c9-4b19-a7ba-06e7ab122ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6818","nonce":3514139178},{"type":"v1","addr":"10.20.193.27:6819","nonce":3514139178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6820","nonce":3514139178},{"type":"v1","addr":"10.20.193.27:6821","nonce":3514139178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6824","nonce":3514139178},{"type":"v1","addr":"10.20.193.27:6825","nonce":3514139178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6822","nonce":3514139178},{"type":"v1","addr":"10.20.193.27:6823","nonce":3514139178}]},"public_addr":"10.20.193.27:6819/3514139178","cluster_addr":"10.20.193.27:6821/3514139178","heartbeat_back_addr":"10.20.193.27:6825/3514139178","heartbeat_front_addr":"10.20.193.27:6823/3514139178","state":["exists","up"]},{"osd":7,"uuid":"b545439a-3f32-4b0a-bc6a-17f0d8050c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6816","nonce":1405200379},{"type":"v1","addr":"10.20.193.28:6817","nonce":1405200379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6818","nonce":1405200379},{"type":"v1","addr":"10.20.193.28:6819","nonce":1405200379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6822","nonce":1405200379},{"type":"v1","addr":"10.20.193.28:6823","nonce":1405200379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6820","nonce":1405200379},{"type":"v1","addr":"10.20.193.28:6821","nonce":1405200379}]},"public_addr":"10.20.193.28:6817/1405200379","cluster_addr":"10.20.193.28:6819/1405200379","heartbeat_back_addr":"10.20.193.28:6823/1405200379","heartbeat_front_addr":"10.20.193.28:6821/1405200379","state":["exists","up"]},{"osd":8,"uuid":"a7c6bbba-760c-484a-8171-44c70f7d9c89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6816","nonce":1331562104},{"type":"v1","addr":"10.20.193.107:6817","nonce":1331562104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6818","nonce":1331562104},{"type":"v1","addr":"10.20.193.107:6819","nonce":1331562104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6822","nonce":1331562104},{"type":"v1","addr":"10.20.193.107:6823","nonce":1331562104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6820","nonce":1331562104},{"type":"v1","addr":"10.20.193.107:6821","nonce":1331562104}]},"public_addr":"10.20.193.107:6817/1331562104","cluster_addr":"10.20.193.107:6819/1331562104","heartbeat_back_addr":"10.20.193.107:6823/1331562104","heartbeat_front_addr":"10.20.193.107:6821/1331562104","state":["exists","up"]},{"osd":9,"uuid":"72cbd422-3c3a-4904-9075-f543671d4f48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6826","nonce":1444927853},{"type":"v1","addr":"10.20.193.27:6827","nonce":1444927853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6828","nonce":1444927853},{"type":"v1","addr":"10.20.193.27:6829","nonce":1444927853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6832","nonce":1444927853},{"type":"v1","addr":"10.20.193.27:6833","nonce":1444927853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6830","nonce":1444927853},{"type":"v1","addr":"10.20.193.27:6831","nonce":1444927853}]},"public_addr":"10.20.193.27:6827/1444927853","cluster_addr":"10.20.193.27:6829/1444927853","heartbeat_back_addr":"10.20.193.27:6833/1444927853","heartbeat_front_addr":"10.20.193.27:6831/1444927853","state":["exists","up"]},{"osd":10,"uuid":"4bb7b75e-adf7-4c38-9343-dba513b57ed7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6824","nonce":2255755797},{"type":"v1","addr":"10.20.193.28:6825","nonce":2255755797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6826","nonce":2255755797},{"type":"v1","addr":"10.20.193.28:6827","nonce":2255755797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6830","nonce":2255755797},{"type":"v1","addr":"10.20.193.28:6831","nonce":2255755797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6828","nonce":2255755797},{"type":"v1","addr":"10.20.193.28:6829","nonce":2255755797}]},"public_addr":"10.20.193.28:6825/2255755797","cluster_addr":"10.20.193.28:6827/2255755797","heartbeat_back_addr":"10.20.193.28:6831/2255755797","heartbeat_front_addr":"10.20.193.28:6829/2255755797","state":["exists","up"]},{"osd":11,"uuid":"115fe8c6-77fc-41dd-ade0-db8eea024974","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6824","nonce":1496433214},{"type":"v1","addr":"10.20.193.107:6825","nonce":1496433214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6826","nonce":1496433214},{"type":"v1","addr":"10.20.193.107:6827","nonce":1496433214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6830","nonce":1496433214},{"type":"v1","addr":"10.20.193.107:6831","nonce":1496433214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6828","nonce":1496433214},{"type":"v1","addr":"10.20.193.107:6829","nonce":1496433214}]},"public_addr":"10.20.193.107:6825/1496433214","cluster_addr":"10.20.193.107:6827/1496433214","heartbeat_back_addr":"10.20.193.107:6831/1496433214","heartbeat_front_addr":"10.20.193.107:6829/1496433214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:20:37.596432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:20:44.867765+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:20:51.765273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:20:58.994342+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:06.232104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:13.195531+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:20.453172+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:27.721147+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:35.051979+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:42.463998+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:49.889642+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:57.167778+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"10.20.193.27:0/1486586326":"2026-02-14T02:19:56.309429+0000","10.20.193.27:6801/3576730225":"2026-02-14T02:19:56.309429+0000","10.20.193.27:6800/3576730225":"2026-02-14T02:19:56.309429+0000","10.20.193.27:0/2609227888":"2026-02-14T02:19:56.309429+0000","10.20.193.27:0/1389740324":"2026-02-14T02:19:56.309429+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-13T02:22:17.398 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.395+0000 7f84e96ac6c0 1 -- 10.20.193.27:0/653159202 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f84b80b5680 msgr2=0x7f84b80d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:17.398 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.395+0000 7f84e96ac6c0 1 --2- 10.20.193.27:0/653159202 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f84b80b5680 0x7f84b80d5b30 secure :-1 s=READY pgs=166 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f84cc005980 tx=0x7f84cc005910 comp rx=0 tx=0).stop 2026-02-13T02:22:17.399 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.395+0000 7f84e96ac6c0 1 -- 10.20.193.27:0/653159202 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f84e410f230 msgr2=0x7f84e411d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:17.399 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.395+0000 7f84e96ac6c0 1 --2- 10.20.193.27:0/653159202 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f84e410f230 0x7f84e411d320 secure :-1 s=READY pgs=220 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f84d802f230 tx=0x7f84d802fd40 comp rx=0 tx=0).stop 2026-02-13T02:22:17.399 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.395+0000 7f84e96ac6c0 1 -- 10.20.193.27:0/653159202 shutdown_connections 2026-02-13T02:22:17.399 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.395+0000 7f84e96ac6c0 1 --2- 10.20.193.27:0/653159202 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f84b80b5680 0x7f84b80d5b30 unknown :-1 s=CLOSED pgs=166 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:17.399 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.395+0000 7f84e96ac6c0 1 --2- 10.20.193.27:0/653159202 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f84e4122c20 0x7f84e41230c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:17.399 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.395+0000 7f84e96ac6c0 1 --2- 10.20.193.27:0/653159202 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f84e4115440 0x7f84e411d860 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:17.399 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.395+0000 7f84e96ac6c0 1 --2- 10.20.193.27:0/653159202 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f84e410f230 0x7f84e411d320 unknown :-1 s=CLOSED pgs=220 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:17.399 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.395+0000 7f84e96ac6c0 1 -- 10.20.193.27:0/653159202 >> 10.20.193.27:0/653159202 conn(0x7f84e40ecb30 msgr2=0x7f84e410cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:17.399 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.395+0000 7f84e96ac6c0 1 -- 10.20.193.27:0/653159202 shutdown_connections 2026-02-13T02:22:17.399 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.395+0000 7f84e96ac6c0 1 -- 10.20.193.27:0/653159202 wait complete. 2026-02-13T02:22:17.440 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd dump --format=json 2026-02-13T02:22:17.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:17 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1897077955' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2026-02-13T02:22:17.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:17 trial028 ceph-mon[26701]: osdmap e68: 12 total, 12 up, 12 in 2026-02-13T02:22:17.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:17 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1479855520' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2026-02-13T02:22:17.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:17 trial028 ceph-mon[26701]: daemon mds.b assigned to filesystem cephfs as rank 0 2026-02-13T02:22:17.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:17 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1897077955' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2026-02-13T02:22:17.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:17 trial107 ceph-mon[27422]: osdmap e68: 12 total, 12 up, 12 in 2026-02-13T02:22:17.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:17 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1479855520' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2026-02-13T02:22:17.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:17 trial107 ceph-mon[27422]: daemon mds.b assigned to filesystem cephfs as rank 0 2026-02-13T02:22:17.557 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:17.680 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.676+0000 7f8522ae96c0 1 -- 10.20.193.27:0/851475145 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f851c1131c0 msgr2=0x7f851c173b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:17.680 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.676+0000 7f8522ae96c0 1 --2- 10.20.193.27:0/851475145 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f851c1131c0 0x7f851c173b60 secure :-1 s=READY pgs=221 gs=1 cs=0 l=1 c_cookie=1274261665264513917 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f851800aa70 tx=0x7f851802fde0 comp rx=0 tx=0).stop 2026-02-13T02:22:17.681 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.676+0000 7f8522ae96c0 1 -- 10.20.193.27:0/851475145 shutdown_connections 2026-02-13T02:22:17.681 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.676+0000 7f8522ae96c0 1 --2- 10.20.193.27:0/851475145 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f851c1131c0 0x7f851c173b60 unknown :-1 s=CLOSED pgs=221 gs=1 cs=0 l=1 c_cookie=1274261665264513917 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:17.681 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.676+0000 7f8522ae96c0 1 --2- 10.20.193.27:0/851475145 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f851c10d9a0 0x7f851c1129e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:17.681 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.676+0000 7f8522ae96c0 1 --2- 10.20.193.27:0/851475145 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f851c10cfb0 0x7f851c10d380 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:17.681 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.676+0000 7f8522ae96c0 1 -- 10.20.193.27:0/851475145 >> 10.20.193.27:0/851475145 conn(0x7f851c0ea8b0 msgr2=0x7f851c10acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:17.681 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.676+0000 7f8522ae96c0 1 -- 10.20.193.27:0/851475145 shutdown_connections 2026-02-13T02:22:17.681 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.676+0000 7f8522ae96c0 1 -- 10.20.193.27:0/851475145 wait complete. 2026-02-13T02:22:17.681 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.676+0000 7f8522ae96c0 1 Processor -- start 2026-02-13T02:22:17.681 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.676+0000 7f8522ae96c0 1 -- start start 2026-02-13T02:22:17.681 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.676+0000 7f8522ae96c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f851c10cfb0 0x7f851c0f4ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:17.682 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.676+0000 7f8522ae96c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f851c1131c0 0x7f851c173a60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:17.682 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.676+0000 7f8522ae96c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f851c0f52d0 0x7f851c0fa5a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:17.682 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.676+0000 7f8522ae96c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f851c0f59c0 con 0x7f851c0f52d0 2026-02-13T02:22:17.682 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.676+0000 7f8522ae96c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f851c0faae0 con 0x7f851c1131c0 2026-02-13T02:22:17.682 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.676+0000 7f8522ae96c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f851c0faca0 con 0x7f851c10cfb0 2026-02-13T02:22:17.682 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.676+0000 7f852105b6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f851c0f52d0 0x7f851c0fa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:17.682 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.676+0000 7f852105b6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f851c0f52d0 0x7f851c0fa5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55174/0 (socket says 10.20.193.27:55174) 2026-02-13T02:22:17.682 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.676+0000 7f852105b6c0 1 -- 10.20.193.27:0/3699543988 learned_addr learned my addr 10.20.193.27:0/3699543988 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:17.682 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.676+0000 7f852085a6c0 1 --2- 10.20.193.27:0/3699543988 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f851c10cfb0 0x7f851c0f4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:17.682 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.677+0000 7f852105b6c0 1 -- 10.20.193.27:0/3699543988 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f851c10cfb0 msgr2=0x7f851c0f4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:17.682 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.677+0000 7f852105b6c0 1 --2- 10.20.193.27:0/3699543988 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f851c10cfb0 0x7f851c0f4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:17.682 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.677+0000 7f852105b6c0 1 -- 10.20.193.27:0/3699543988 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f851c1131c0 msgr2=0x7f851c173a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:17.683 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.677+0000 7f852105b6c0 1 --2- 10.20.193.27:0/3699543988 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f851c1131c0 0x7f851c173a60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:17.683 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.677+0000 7f852105b6c0 1 -- 10.20.193.27:0/3699543988 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851800a660 con 0x7f851c0f52d0 2026-02-13T02:22:17.683 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.677+0000 7f852085a6c0 1 --2- 10.20.193.27:0/3699543988 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f851c10cfb0 0x7f851c0f4ca0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:22:17.683 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.677+0000 7f852105b6c0 1 --2- 10.20.193.27:0/3699543988 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f851c0f52d0 0x7f851c0fa5a0 secure :-1 s=READY pgs=222 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f85180054b0 tx=0x7f85180054e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:17.683 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.677+0000 7f8511ffb6c0 1 -- 10.20.193.27:0/3699543988 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8518003a30 con 0x7f851c0f52d0 2026-02-13T02:22:17.683 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.677+0000 7f8511ffb6c0 1 -- 10.20.193.27:0/3699543988 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f8518003b90 con 0x7f851c0f52d0 2026-02-13T02:22:17.683 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.677+0000 7f8522ae96c0 1 -- 10.20.193.27:0/3699543988 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f851c0faf20 con 0x7f851c0f52d0 2026-02-13T02:22:17.683 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.677+0000 7f8511ffb6c0 1 -- 10.20.193.27:0/3699543988 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f85180382f0 con 0x7f851c0f52d0 2026-02-13T02:22:17.683 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.677+0000 7f8522ae96c0 1 -- 10.20.193.27:0/3699543988 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f851c0fb350 con 0x7f851c0f52d0 2026-02-13T02:22:17.683 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.677+0000 7f8522ae96c0 1 -- 10.20.193.27:0/3699543988 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f851c10e280 con 0x7f851c0f52d0 2026-02-13T02:22:17.684 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.678+0000 7f8511ffb6c0 1 -- 10.20.193.27:0/3699543988 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7f8518008260 con 0x7f851c0f52d0 2026-02-13T02:22:17.684 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.679+0000 7f8511ffb6c0 1 --2- 10.20.193.27:0/3699543988 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f84f80b5600 0x7f84f80d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:17.684 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.679+0000 7f8511ffb6c0 1 -- 10.20.193.27:0/3699543988 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7762+0+0 (secure 0 0 0) 0x7f8518108c60 con 0x7f851c0f52d0 2026-02-13T02:22:17.684 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.679+0000 7f852085a6c0 1 --2- 10.20.193.27:0/3699543988 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f84f80b5600 0x7f84f80d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:17.685 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.679+0000 7f852085a6c0 1 --2- 10.20.193.27:0/3699543988 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f84f80b5600 0x7f84f80d5ab0 secure :-1 s=READY pgs=167 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f850c005dc0 tx=0x7f850c005f90 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:17.685 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.679+0000 7f8511ffb6c0 1 -- 10.20.193.27:0/3699543988 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f85180cb7e0 con 0x7f851c0f52d0 2026-02-13T02:22:17.772 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.770+0000 7f8522ae96c0 1 -- 10.20.193.27:0/3699543988 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f851c0f5e20 con 0x7f851c0f52d0 2026-02-13T02:22:17.773 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.770+0000 7f8511ffb6c0 1 -- 10.20.193.27:0/3699543988 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+22015 (secure 0 0 0) 0x7f85180ce990 con 0x7f851c0f52d0 2026-02-13T02:22:17.773 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:22:17.774 INFO:teuthology.orchestra.run.trial027.stdout:{"epoch":69,"fsid":"707dbb29-0882-11f1-9262-d404e6e7d460","created":"2026-02-13T02:19:50.420575+0000","modified":"2026-02-13T02:22:16.406005+0000","last_up_change":"2026-02-13T02:21:58.960121+0000","last_in_change":"2026-02-13T02:21:50.695981+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-13T02:20:54.702258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2026-02-13T02:22:14.280021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2026-02-13T02:22:15.383271+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1a7af29f-2999-4be3-a38e-3fc7feb9f2f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":7,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6802","nonce":832083434},{"type":"v1","addr":"10.20.193.27:6803","nonce":832083434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6804","nonce":832083434},{"type":"v1","addr":"10.20.193.27:6805","nonce":832083434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6808","nonce":832083434},{"type":"v1","addr":"10.20.193.27:6809","nonce":832083434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6806","nonce":832083434},{"type":"v1","addr":"10.20.193.27:6807","nonce":832083434}]},"public_addr":"10.20.193.27:6803/832083434","cluster_addr":"10.20.193.27:6805/832083434","heartbeat_back_addr":"10.20.193.27:6809/832083434","heartbeat_front_addr":"10.20.193.27:6807/832083434","state":["exists","up"]},{"osd":1,"uuid":"8c0ad8cf-4150-43f6-8a9c-258b0ebf6a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6800","nonce":4070516162},{"type":"v1","addr":"10.20.193.28:6801","nonce":4070516162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6802","nonce":4070516162},{"type":"v1","addr":"10.20.193.28:6803","nonce":4070516162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6806","nonce":4070516162},{"type":"v1","addr":"10.20.193.28:6807","nonce":4070516162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6804","nonce":4070516162},{"type":"v1","addr":"10.20.193.28:6805","nonce":4070516162}]},"public_addr":"10.20.193.28:6801/4070516162","cluster_addr":"10.20.193.28:6803/4070516162","heartbeat_back_addr":"10.20.193.28:6807/4070516162","heartbeat_front_addr":"10.20.193.28:6805/4070516162","state":["exists","up"]},{"osd":2,"uuid":"a7813078-94d6-4559-a89a-5525161ab5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6800","nonce":4075625103},{"type":"v1","addr":"10.20.193.107:6801","nonce":4075625103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6802","nonce":4075625103},{"type":"v1","addr":"10.20.193.107:6803","nonce":4075625103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6806","nonce":4075625103},{"type":"v1","addr":"10.20.193.107:6807","nonce":4075625103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6804","nonce":4075625103},{"type":"v1","addr":"10.20.193.107:6805","nonce":4075625103}]},"public_addr":"10.20.193.107:6801/4075625103","cluster_addr":"10.20.193.107:6803/4075625103","heartbeat_back_addr":"10.20.193.107:6807/4075625103","heartbeat_front_addr":"10.20.193.107:6805/4075625103","state":["exists","up"]},{"osd":3,"uuid":"268f9714-faed-452b-ae34-3240fa23153a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6810","nonce":382071678},{"type":"v1","addr":"10.20.193.27:6811","nonce":382071678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6812","nonce":382071678},{"type":"v1","addr":"10.20.193.27:6813","nonce":382071678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6816","nonce":382071678},{"type":"v1","addr":"10.20.193.27:6817","nonce":382071678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6814","nonce":382071678},{"type":"v1","addr":"10.20.193.27:6815","nonce":382071678}]},"public_addr":"10.20.193.27:6811/382071678","cluster_addr":"10.20.193.27:6813/382071678","heartbeat_back_addr":"10.20.193.27:6817/382071678","heartbeat_front_addr":"10.20.193.27:6815/382071678","state":["exists","up"]},{"osd":4,"uuid":"4beaec0d-7b39-4bb0-9770-3dc4c848b106","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6808","nonce":711804032},{"type":"v1","addr":"10.20.193.28:6809","nonce":711804032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6810","nonce":711804032},{"type":"v1","addr":"10.20.193.28:6811","nonce":711804032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6814","nonce":711804032},{"type":"v1","addr":"10.20.193.28:6815","nonce":711804032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6812","nonce":711804032},{"type":"v1","addr":"10.20.193.28:6813","nonce":711804032}]},"public_addr":"10.20.193.28:6809/711804032","cluster_addr":"10.20.193.28:6811/711804032","heartbeat_back_addr":"10.20.193.28:6815/711804032","heartbeat_front_addr":"10.20.193.28:6813/711804032","state":["exists","up"]},{"osd":5,"uuid":"bb16754e-66f2-4951-9f31-d1dde591ec92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6808","nonce":4246100128},{"type":"v1","addr":"10.20.193.107:6809","nonce":4246100128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6810","nonce":4246100128},{"type":"v1","addr":"10.20.193.107:6811","nonce":4246100128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6814","nonce":4246100128},{"type":"v1","addr":"10.20.193.107:6815","nonce":4246100128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6812","nonce":4246100128},{"type":"v1","addr":"10.20.193.107:6813","nonce":4246100128}]},"public_addr":"10.20.193.107:6809/4246100128","cluster_addr":"10.20.193.107:6811/4246100128","heartbeat_back_addr":"10.20.193.107:6815/4246100128","heartbeat_front_addr":"10.20.193.107:6813/4246100128","state":["exists","up"]},{"osd":6,"uuid":"318afff7-b7c9-4b19-a7ba-06e7ab122ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6818","nonce":3514139178},{"type":"v1","addr":"10.20.193.27:6819","nonce":3514139178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6820","nonce":3514139178},{"type":"v1","addr":"10.20.193.27:6821","nonce":3514139178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6824","nonce":3514139178},{"type":"v1","addr":"10.20.193.27:6825","nonce":3514139178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6822","nonce":3514139178},{"type":"v1","addr":"10.20.193.27:6823","nonce":3514139178}]},"public_addr":"10.20.193.27:6819/3514139178","cluster_addr":"10.20.193.27:6821/3514139178","heartbeat_back_addr":"10.20.193.27:6825/3514139178","heartbeat_front_addr":"10.20.193.27:6823/3514139178","state":["exists","up"]},{"osd":7,"uuid":"b545439a-3f32-4b0a-bc6a-17f0d8050c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6816","nonce":1405200379},{"type":"v1","addr":"10.20.193.28:6817","nonce":1405200379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6818","nonce":1405200379},{"type":"v1","addr":"10.20.193.28:6819","nonce":1405200379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6822","nonce":1405200379},{"type":"v1","addr":"10.20.193.28:6823","nonce":1405200379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6820","nonce":1405200379},{"type":"v1","addr":"10.20.193.28:6821","nonce":1405200379}]},"public_addr":"10.20.193.28:6817/1405200379","cluster_addr":"10.20.193.28:6819/1405200379","heartbeat_back_addr":"10.20.193.28:6823/1405200379","heartbeat_front_addr":"10.20.193.28:6821/1405200379","state":["exists","up"]},{"osd":8,"uuid":"a7c6bbba-760c-484a-8171-44c70f7d9c89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6816","nonce":1331562104},{"type":"v1","addr":"10.20.193.107:6817","nonce":1331562104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6818","nonce":1331562104},{"type":"v1","addr":"10.20.193.107:6819","nonce":1331562104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6822","nonce":1331562104},{"type":"v1","addr":"10.20.193.107:6823","nonce":1331562104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6820","nonce":1331562104},{"type":"v1","addr":"10.20.193.107:6821","nonce":1331562104}]},"public_addr":"10.20.193.107:6817/1331562104","cluster_addr":"10.20.193.107:6819/1331562104","heartbeat_back_addr":"10.20.193.107:6823/1331562104","heartbeat_front_addr":"10.20.193.107:6821/1331562104","state":["exists","up"]},{"osd":9,"uuid":"72cbd422-3c3a-4904-9075-f543671d4f48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6826","nonce":1444927853},{"type":"v1","addr":"10.20.193.27:6827","nonce":1444927853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6828","nonce":1444927853},{"type":"v1","addr":"10.20.193.27:6829","nonce":1444927853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6832","nonce":1444927853},{"type":"v1","addr":"10.20.193.27:6833","nonce":1444927853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6830","nonce":1444927853},{"type":"v1","addr":"10.20.193.27:6831","nonce":1444927853}]},"public_addr":"10.20.193.27:6827/1444927853","cluster_addr":"10.20.193.27:6829/1444927853","heartbeat_back_addr":"10.20.193.27:6833/1444927853","heartbeat_front_addr":"10.20.193.27:6831/1444927853","state":["exists","up"]},{"osd":10,"uuid":"4bb7b75e-adf7-4c38-9343-dba513b57ed7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6824","nonce":2255755797},{"type":"v1","addr":"10.20.193.28:6825","nonce":2255755797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6826","nonce":2255755797},{"type":"v1","addr":"10.20.193.28:6827","nonce":2255755797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6830","nonce":2255755797},{"type":"v1","addr":"10.20.193.28:6831","nonce":2255755797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6828","nonce":2255755797},{"type":"v1","addr":"10.20.193.28:6829","nonce":2255755797}]},"public_addr":"10.20.193.28:6825/2255755797","cluster_addr":"10.20.193.28:6827/2255755797","heartbeat_back_addr":"10.20.193.28:6831/2255755797","heartbeat_front_addr":"10.20.193.28:6829/2255755797","state":["exists","up"]},{"osd":11,"uuid":"115fe8c6-77fc-41dd-ade0-db8eea024974","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6824","nonce":1496433214},{"type":"v1","addr":"10.20.193.107:6825","nonce":1496433214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6826","nonce":1496433214},{"type":"v1","addr":"10.20.193.107:6827","nonce":1496433214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6830","nonce":1496433214},{"type":"v1","addr":"10.20.193.107:6831","nonce":1496433214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6828","nonce":1496433214},{"type":"v1","addr":"10.20.193.107:6829","nonce":1496433214}]},"public_addr":"10.20.193.107:6825/1496433214","cluster_addr":"10.20.193.107:6827/1496433214","heartbeat_back_addr":"10.20.193.107:6831/1496433214","heartbeat_front_addr":"10.20.193.107:6829/1496433214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:20:37.596432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:20:44.867765+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:20:51.765273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:20:58.994342+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:06.232104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:13.195531+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:20.453172+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:27.721147+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:35.051979+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:42.463998+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:49.889642+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:57.167778+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"10.20.193.27:0/1486586326":"2026-02-14T02:19:56.309429+0000","10.20.193.27:6801/3576730225":"2026-02-14T02:19:56.309429+0000","10.20.193.27:6800/3576730225":"2026-02-14T02:19:56.309429+0000","10.20.193.27:0/2609227888":"2026-02-14T02:19:56.309429+0000","10.20.193.27:0/1389740324":"2026-02-14T02:19:56.309429+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-13T02:22:17.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.772+0000 7f8522ae96c0 1 -- 10.20.193.27:0/3699543988 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f84f80b5600 msgr2=0x7f84f80d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:17.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.772+0000 7f8522ae96c0 1 --2- 10.20.193.27:0/3699543988 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f84f80b5600 0x7f84f80d5ab0 secure :-1 s=READY pgs=167 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f850c005dc0 tx=0x7f850c005f90 comp rx=0 tx=0).stop 2026-02-13T02:22:17.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.772+0000 7f8522ae96c0 1 -- 10.20.193.27:0/3699543988 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f851c0f52d0 msgr2=0x7f851c0fa5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:17.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.772+0000 7f8522ae96c0 1 --2- 10.20.193.27:0/3699543988 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f851c0f52d0 0x7f851c0fa5a0 secure :-1 s=READY pgs=222 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f85180054b0 tx=0x7f85180054e0 comp rx=0 tx=0).stop 2026-02-13T02:22:17.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.772+0000 7f8522ae96c0 1 -- 10.20.193.27:0/3699543988 shutdown_connections 2026-02-13T02:22:17.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.772+0000 7f8522ae96c0 1 --2- 10.20.193.27:0/3699543988 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f84f80b5600 0x7f84f80d5ab0 unknown :-1 s=CLOSED pgs=167 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:17.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.772+0000 7f8522ae96c0 1 --2- 10.20.193.27:0/3699543988 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f851c0f52d0 0x7f851c0fa5a0 unknown :-1 s=CLOSED pgs=222 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:17.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.772+0000 7f8522ae96c0 1 --2- 10.20.193.27:0/3699543988 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f851c1131c0 0x7f851c173a60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:17.776 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.772+0000 7f8522ae96c0 1 --2- 10.20.193.27:0/3699543988 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f851c10cfb0 0x7f851c0f4ca0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:17.776 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.772+0000 7f8522ae96c0 1 -- 10.20.193.27:0/3699543988 >> 10.20.193.27:0/3699543988 conn(0x7f851c0ea8b0 msgr2=0x7f851c110c90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:17.776 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.772+0000 7f8522ae96c0 1 -- 10.20.193.27:0/3699543988 shutdown_connections 2026-02-13T02:22:17.776 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:17.772+0000 7f8522ae96c0 1 -- 10.20.193.27:0/3699543988 wait complete. 2026-02-13T02:22:17.817 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph fs set cephfs standby_count_wanted 0 2026-02-13T02:22:17.933 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:18.054 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.050+0000 7ff658b136c0 1 -- 10.20.193.27:0/915207420 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff654114d90 msgr2=0x7ff65415dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:18.054 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.050+0000 7ff658b136c0 1 --2- 10.20.193.27:0/915207420 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff654114d90 0x7ff65415dc70 secure :-1 s=READY pgs=223 gs=1 cs=0 l=1 c_cookie=5343203958480784685 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff63c009970 tx=0x7ff63c02ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:18.055 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.050+0000 7ff658b136c0 1 -- 10.20.193.27:0/915207420 shutdown_connections 2026-02-13T02:22:18.055 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.050+0000 7ff658b136c0 1 --2- 10.20.193.27:0/915207420 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff65415e1b0 0x7ff65417e590 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:18.055 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.050+0000 7ff658b136c0 1 --2- 10.20.193.27:0/915207420 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff654114d90 0x7ff65415dc70 unknown :-1 s=CLOSED pgs=223 gs=1 cs=0 l=1 c_cookie=5343203958480784685 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:18.055 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.050+0000 7ff658b136c0 1 --2- 10.20.193.27:0/915207420 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff654114480 0x7ff654114850 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:18.055 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.050+0000 7ff658b136c0 1 -- 10.20.193.27:0/915207420 >> 10.20.193.27:0/915207420 conn(0x7ff65405cef0 msgr2=0x7ff65405d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:18.055 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.050+0000 7ff658b136c0 1 -- 10.20.193.27:0/915207420 shutdown_connections 2026-02-13T02:22:18.055 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.050+0000 7ff658b136c0 1 -- 10.20.193.27:0/915207420 wait complete. 2026-02-13T02:22:18.055 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.050+0000 7ff658b136c0 1 Processor -- start 2026-02-13T02:22:18.055 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.050+0000 7ff658b136c0 1 -- start start 2026-02-13T02:22:18.055 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.050+0000 7ff658b136c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff654114480 0x7ff6540fe270 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:18.056 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.050+0000 7ff658b136c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff654114d90 0x7ff6540fe7b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:18.056 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.050+0000 7ff658b136c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff65415e1b0 0x7ff654103b70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:18.056 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.050+0000 7ff658b136c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff6541041c0 con 0x7ff65415e1b0 2026-02-13T02:22:18.056 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.050+0000 7ff658b136c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff654104330 con 0x7ff654114d90 2026-02-13T02:22:18.056 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.050+0000 7ff658b136c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff6541044f0 con 0x7ff654114480 2026-02-13T02:22:18.056 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.051+0000 7ff652d766c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff65415e1b0 0x7ff654103b70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:18.056 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.051+0000 7ff652d766c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff65415e1b0 0x7ff654103b70 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55206/0 (socket says 10.20.193.27:55206) 2026-02-13T02:22:18.056 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.051+0000 7ff652d766c0 1 -- 10.20.193.27:0/1998777205 learned_addr learned my addr 10.20.193.27:0/1998777205 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:18.056 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.051+0000 7ff6525756c0 1 --2- 10.20.193.27:0/1998777205 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff654114480 0x7ff6540fe270 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:18.056 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.051+0000 7ff651d746c0 1 --2- 10.20.193.27:0/1998777205 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff654114d90 0x7ff6540fe7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:18.056 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.051+0000 7ff652d766c0 1 -- 10.20.193.27:0/1998777205 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff654114480 msgr2=0x7ff6540fe270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:18.056 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.051+0000 7ff652d766c0 1 --2- 10.20.193.27:0/1998777205 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff654114480 0x7ff6540fe270 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:18.057 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.051+0000 7ff652d766c0 1 -- 10.20.193.27:0/1998777205 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff654114d90 msgr2=0x7ff6540fe7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:18.057 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.051+0000 7ff652d766c0 1 --2- 10.20.193.27:0/1998777205 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff654114d90 0x7ff6540fe7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:18.057 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.051+0000 7ff652d766c0 1 -- 10.20.193.27:0/1998777205 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63c009590 con 0x7ff65415e1b0 2026-02-13T02:22:18.057 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.051+0000 7ff6525756c0 1 --2- 10.20.193.27:0/1998777205 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff654114480 0x7ff6540fe270 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:22:18.057 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.051+0000 7ff652d766c0 1 --2- 10.20.193.27:0/1998777205 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff65415e1b0 0x7ff654103b70 secure :-1 s=READY pgs=224 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff64400c6c0 tx=0x7ff64400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:18.057 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.051+0000 7ff63b7fe6c0 1 -- 10.20.193.27:0/1998777205 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff644003c40 con 0x7ff65415e1b0 2026-02-13T02:22:18.057 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.051+0000 7ff63b7fe6c0 1 -- 10.20.193.27:0/1998777205 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff64400ce70 con 0x7ff65415e1b0 2026-02-13T02:22:18.057 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.051+0000 7ff658b136c0 1 -- 10.20.193.27:0/1998777205 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6541047d0 con 0x7ff65415e1b0 2026-02-13T02:22:18.057 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.051+0000 7ff63b7fe6c0 1 -- 10.20.193.27:0/1998777205 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff64400dbe0 con 0x7ff65415e1b0 2026-02-13T02:22:18.057 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.051+0000 7ff658b136c0 1 -- 10.20.193.27:0/1998777205 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff65415adf0 con 0x7ff65415e1b0 2026-02-13T02:22:18.057 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.051+0000 7ff658b136c0 1 -- 10.20.193.27:0/1998777205 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff654057030 con 0x7ff65415e1b0 2026-02-13T02:22:18.058 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.052+0000 7ff63b7fe6c0 1 -- 10.20.193.27:0/1998777205 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7ff644005030 con 0x7ff65415e1b0 2026-02-13T02:22:18.059 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.053+0000 7ff63b7fe6c0 1 --2- 10.20.193.27:0/1998777205 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff62c0b5680 0x7ff62c0d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:18.059 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.053+0000 7ff6525756c0 1 --2- 10.20.193.27:0/1998777205 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff62c0b5680 0x7ff62c0d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:18.059 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.053+0000 7ff63b7fe6c0 1 -- 10.20.193.27:0/1998777205 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7ff6440e47b0 con 0x7ff65415e1b0 2026-02-13T02:22:18.059 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.053+0000 7ff6525756c0 1 --2- 10.20.193.27:0/1998777205 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff62c0b5680 0x7ff62c0d5b30 secure :-1 s=READY pgs=168 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff648004750 tx=0x7ff648009630 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:18.059 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.053+0000 7ff63b7fe6c0 1 -- 10.20.193.27:0/1998777205 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7ff6440a7330 con 0x7ff65415e1b0 2026-02-13T02:22:18.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:18 trial027 ceph-mon[24567]: pgmap v182: 97 pgs: 16 creating+peering, 70 unknown, 11 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:18.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:18 trial027 ceph-mon[24567]: osdmap e69: 12 total, 12 up, 12 in 2026-02-13T02:22:18.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:18 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1479855520' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2026-02-13T02:22:18.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:18 trial027 ceph-mon[24567]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2026-02-13T02:22:18.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:18 trial027 ceph-mon[24567]: daemon mds.b is now active in filesystem cephfs as rank 0 2026-02-13T02:22:18.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:18 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/653159202' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-13T02:22:18.159 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:18 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3699543988' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-13T02:22:18.171 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:18.169+0000 7ff658b136c0 1 -- 10.20.193.27:0/1998777205 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff654114850 con 0x7ff65415e1b0 2026-02-13T02:22:18.402 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:18 trial107 ceph-mon[27422]: pgmap v182: 97 pgs: 16 creating+peering, 70 unknown, 11 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:18.402 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:18 trial107 ceph-mon[27422]: osdmap e69: 12 total, 12 up, 12 in 2026-02-13T02:22:18.402 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:18 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1479855520' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2026-02-13T02:22:18.402 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:18 trial107 ceph-mon[27422]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2026-02-13T02:22:18.402 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:18 trial107 ceph-mon[27422]: daemon mds.b is now active in filesystem cephfs as rank 0 2026-02-13T02:22:18.402 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:18 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/653159202' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-13T02:22:18.402 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:18 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3699543988' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-13T02:22:18.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:18 trial028 ceph-mon[26701]: pgmap v182: 97 pgs: 16 creating+peering, 70 unknown, 11 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-13T02:22:18.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:18 trial028 ceph-mon[26701]: osdmap e69: 12 total, 12 up, 12 in 2026-02-13T02:22:18.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:18 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1479855520' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2026-02-13T02:22:18.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:18 trial028 ceph-mon[26701]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2026-02-13T02:22:18.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:18 trial028 ceph-mon[26701]: daemon mds.b is now active in filesystem cephfs as rank 0 2026-02-13T02:22:18.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:18 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/653159202' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-13T02:22:18.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:18 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3699543988' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-13T02:22:19.022 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.020+0000 7ff63b7fe6c0 1 -- 10.20.193.27:0/1998777205 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7ff6440aa4e0 con 0x7ff65415e1b0 2026-02-13T02:22:19.024 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.022+0000 7ff658b136c0 1 -- 10.20.193.27:0/1998777205 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff62c0b5680 msgr2=0x7ff62c0d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:19.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.022+0000 7ff658b136c0 1 --2- 10.20.193.27:0/1998777205 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff62c0b5680 0x7ff62c0d5b30 secure :-1 s=READY pgs=168 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff648004750 tx=0x7ff648009630 comp rx=0 tx=0).stop 2026-02-13T02:22:19.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.022+0000 7ff658b136c0 1 -- 10.20.193.27:0/1998777205 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff65415e1b0 msgr2=0x7ff654103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:19.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.022+0000 7ff658b136c0 1 --2- 10.20.193.27:0/1998777205 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff65415e1b0 0x7ff654103b70 secure :-1 s=READY pgs=224 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff64400c6c0 tx=0x7ff64400cb90 comp rx=0 tx=0).stop 2026-02-13T02:22:19.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.022+0000 7ff658b136c0 1 -- 10.20.193.27:0/1998777205 shutdown_connections 2026-02-13T02:22:19.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.022+0000 7ff658b136c0 1 --2- 10.20.193.27:0/1998777205 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff62c0b5680 0x7ff62c0d5b30 unknown :-1 s=CLOSED pgs=168 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:19.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.022+0000 7ff658b136c0 1 --2- 10.20.193.27:0/1998777205 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff65415e1b0 0x7ff654103b70 unknown :-1 s=CLOSED pgs=224 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:19.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.022+0000 7ff658b136c0 1 --2- 10.20.193.27:0/1998777205 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff654114d90 0x7ff6540fe7b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:19.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.022+0000 7ff658b136c0 1 --2- 10.20.193.27:0/1998777205 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff654114480 0x7ff6540fe270 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:19.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.022+0000 7ff658b136c0 1 -- 10.20.193.27:0/1998777205 >> 10.20.193.27:0/1998777205 conn(0x7ff65405cef0 msgr2=0x7ff6540f4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:19.026 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.022+0000 7ff658b136c0 1 -- 10.20.193.27:0/1998777205 shutdown_connections 2026-02-13T02:22:19.026 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.022+0000 7ff658b136c0 1 -- 10.20.193.27:0/1998777205 wait complete. 2026-02-13T02:22:19.065 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['disabled'], 'max_mds': 1, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2026-02-13T02:22:19.065 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph fs set cephfs allow_standby_replay true 2026-02-13T02:22:19.182 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:19.296 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:19 trial027 ceph-mon[24567]: osdmap e70: 12 total, 12 up, 12 in 2026-02-13T02:22:19.297 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:19 trial027 ceph-mon[24567]: mds.? [v2:10.20.193.28:6838/2398862063,v1:10.20.193.28:6839/2398862063] up:active 2026-02-13T02:22:19.297 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:19 trial027 ceph-mon[24567]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2026-02-13T02:22:19.297 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:19 trial027 ceph-mon[24567]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2026-02-13T02:22:19.297 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:19 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1998777205' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2026-02-13T02:22:19.301 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.297+0000 7f9d91a7c6c0 1 -- 10.20.193.27:0/1446693242 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9d8c1131d0 msgr2=0x7f9d8c173b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:19.301 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.297+0000 7f9d91a7c6c0 1 --2- 10.20.193.27:0/1446693242 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9d8c1131d0 0x7f9d8c173b70 secure :-1 s=READY pgs=225 gs=1 cs=0 l=1 c_cookie=4977951596696018937 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9d7c00aa70 tx=0x7f9d7c02fde0 comp rx=0 tx=0).stop 2026-02-13T02:22:19.301 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.297+0000 7f9d91a7c6c0 1 -- 10.20.193.27:0/1446693242 shutdown_connections 2026-02-13T02:22:19.301 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.297+0000 7f9d91a7c6c0 1 --2- 10.20.193.27:0/1446693242 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9d8c1131d0 0x7f9d8c173b70 unknown :-1 s=CLOSED pgs=225 gs=1 cs=0 l=1 c_cookie=4977951596696018937 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:19.302 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.297+0000 7f9d91a7c6c0 1 --2- 10.20.193.27:0/1446693242 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9d8c10bd50 0x7f9d8c1129f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:19.302 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.297+0000 7f9d91a7c6c0 1 --2- 10.20.193.27:0/1446693242 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9d8c10b360 0x7f9d8c10b730 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:19.302 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.297+0000 7f9d91a7c6c0 1 -- 10.20.193.27:0/1446693242 >> 10.20.193.27:0/1446693242 conn(0x7f9d8c062e00 msgr2=0x7f9d8c109660 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:19.302 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.297+0000 7f9d91a7c6c0 1 -- 10.20.193.27:0/1446693242 shutdown_connections 2026-02-13T02:22:19.302 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.297+0000 7f9d91a7c6c0 1 -- 10.20.193.27:0/1446693242 wait complete. 2026-02-13T02:22:19.302 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.297+0000 7f9d91a7c6c0 1 Processor -- start 2026-02-13T02:22:19.302 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.297+0000 7f9d91a7c6c0 1 -- start start 2026-02-13T02:22:19.302 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.297+0000 7f9d91a7c6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9d8c10b360 0x7f9d8c0f4610 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:19.302 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.297+0000 7f9d91a7c6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9d8c1131d0 0x7f9d8c173a70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:19.302 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.297+0000 7f9d91a7c6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9d8c0f4c40 0x7f9d8c0f9f10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:19.302 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.297+0000 7f9d91a7c6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c0f5330 con 0x7f9d8c0f4c40 2026-02-13T02:22:19.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.297+0000 7f9d91a7c6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c0fa450 con 0x7f9d8c1131d0 2026-02-13T02:22:19.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.297+0000 7f9d91a7c6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c0fa610 con 0x7f9d8c10b360 2026-02-13T02:22:19.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.298+0000 7f9d8b7fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9d8c0f4c40 0x7f9d8c0f9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:19.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.298+0000 7f9d8b7fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9d8c0f4c40 0x7f9d8c0f9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55228/0 (socket says 10.20.193.27:55228) 2026-02-13T02:22:19.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.298+0000 7f9d8b7fe6c0 1 -- 10.20.193.27:0/4138477229 learned_addr learned my addr 10.20.193.27:0/4138477229 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:19.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.298+0000 7f9d8affd6c0 1 --2- 10.20.193.27:0/4138477229 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9d8c10b360 0x7f9d8c0f4610 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:19.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.298+0000 7f9d8a7fc6c0 1 --2- 10.20.193.27:0/4138477229 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9d8c1131d0 0x7f9d8c173a70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:19.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.298+0000 7f9d8b7fe6c0 1 -- 10.20.193.27:0/4138477229 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9d8c10b360 msgr2=0x7f9d8c0f4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:19.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.298+0000 7f9d8b7fe6c0 1 --2- 10.20.193.27:0/4138477229 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9d8c10b360 0x7f9d8c0f4610 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:19.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.298+0000 7f9d8b7fe6c0 1 -- 10.20.193.27:0/4138477229 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9d8c1131d0 msgr2=0x7f9d8c173a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:19.303 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.298+0000 7f9d8b7fe6c0 1 --2- 10.20.193.27:0/4138477229 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9d8c1131d0 0x7f9d8c173a70 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:19.304 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.298+0000 7f9d8b7fe6c0 1 -- 10.20.193.27:0/4138477229 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d7c00a660 con 0x7f9d8c0f4c40 2026-02-13T02:22:19.304 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.298+0000 7f9d8b7fe6c0 1 --2- 10.20.193.27:0/4138477229 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9d8c0f4c40 0x7f9d8c0f9f10 secure :-1 s=READY pgs=226 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9d7c0054b0 tx=0x7f9d7c0054e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:19.304 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.298+0000 7f9d90a7a6c0 1 -- 10.20.193.27:0/4138477229 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9d7c003a30 con 0x7f9d8c0f4c40 2026-02-13T02:22:19.304 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.298+0000 7f9d91a7c6c0 1 -- 10.20.193.27:0/4138477229 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d8c0fa890 con 0x7f9d8c0f4c40 2026-02-13T02:22:19.304 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.298+0000 7f9d90a7a6c0 1 -- 10.20.193.27:0/4138477229 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9d7c003b90 con 0x7f9d8c0f4c40 2026-02-13T02:22:19.304 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.298+0000 7f9d90a7a6c0 1 -- 10.20.193.27:0/4138477229 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9d7c0382f0 con 0x7f9d8c0f4c40 2026-02-13T02:22:19.304 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.298+0000 7f9d91a7c6c0 1 -- 10.20.193.27:0/4138477229 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d8c0fad10 con 0x7f9d8c0f4c40 2026-02-13T02:22:19.304 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.298+0000 7f9d91a7c6c0 1 -- 10.20.193.27:0/4138477229 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d8c05f220 con 0x7f9d8c0f4c40 2026-02-13T02:22:19.305 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.300+0000 7f9d90a7a6c0 1 -- 10.20.193.27:0/4138477229 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7f9d7c008260 con 0x7f9d8c0f4c40 2026-02-13T02:22:19.305 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.301+0000 7f9d90a7a6c0 1 --2- 10.20.193.27:0/4138477229 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9d600b5680 0x7f9d600d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:19.305 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.301+0000 7f9d8affd6c0 1 --2- 10.20.193.27:0/4138477229 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9d600b5680 0x7f9d600d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:19.305 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.301+0000 7f9d90a7a6c0 1 -- 10.20.193.27:0/4138477229 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f9d7c108c90 con 0x7f9d8c0f4c40 2026-02-13T02:22:19.305 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.301+0000 7f9d8affd6c0 1 --2- 10.20.193.27:0/4138477229 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9d600b5680 0x7f9d600d5b30 secure :-1 s=READY pgs=169 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9d80005980 tx=0x7f9d80005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:19.306 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.301+0000 7f9d90a7a6c0 1 -- 10.20.193.27:0/4138477229 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f9d7c0cb810 con 0x7f9d8c0f4c40 2026-02-13T02:22:19.421 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:19.419+0000 7f9d91a7c6c0 1 -- 10.20.193.27:0/4138477229 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f9d8c10b730 con 0x7f9d8c0f4c40 2026-02-13T02:22:19.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:19 trial028 ceph-mon[26701]: osdmap e70: 12 total, 12 up, 12 in 2026-02-13T02:22:19.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:19 trial028 ceph-mon[26701]: mds.? [v2:10.20.193.28:6838/2398862063,v1:10.20.193.28:6839/2398862063] up:active 2026-02-13T02:22:19.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:19 trial028 ceph-mon[26701]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2026-02-13T02:22:19.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:19 trial028 ceph-mon[26701]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2026-02-13T02:22:19.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:19 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1998777205' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2026-02-13T02:22:19.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:19 trial107 ceph-mon[27422]: osdmap e70: 12 total, 12 up, 12 in 2026-02-13T02:22:19.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:19 trial107 ceph-mon[27422]: mds.? [v2:10.20.193.28:6838/2398862063,v1:10.20.193.28:6839/2398862063] up:active 2026-02-13T02:22:19.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:19 trial107 ceph-mon[27422]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2026-02-13T02:22:19.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:19 trial107 ceph-mon[27422]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2026-02-13T02:22:19.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:19 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1998777205' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2026-02-13T02:22:20.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.025+0000 7f9d90a7a6c0 1 -- 10.20.193.27:0/4138477229 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v16) ==== 121+0+0 (secure 0 0 0) 0x7f9d7c0ce9c0 con 0x7f9d8c0f4c40 2026-02-13T02:22:20.030 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.027+0000 7f9d91a7c6c0 1 -- 10.20.193.27:0/4138477229 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9d600b5680 msgr2=0x7f9d600d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:20.030 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.027+0000 7f9d91a7c6c0 1 --2- 10.20.193.27:0/4138477229 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9d600b5680 0x7f9d600d5b30 secure :-1 s=READY pgs=169 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9d80005980 tx=0x7f9d80005910 comp rx=0 tx=0).stop 2026-02-13T02:22:20.030 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.027+0000 7f9d91a7c6c0 1 -- 10.20.193.27:0/4138477229 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9d8c0f4c40 msgr2=0x7f9d8c0f9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:20.030 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.027+0000 7f9d91a7c6c0 1 --2- 10.20.193.27:0/4138477229 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9d8c0f4c40 0x7f9d8c0f9f10 secure :-1 s=READY pgs=226 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9d7c0054b0 tx=0x7f9d7c0054e0 comp rx=0 tx=0).stop 2026-02-13T02:22:20.030 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.027+0000 7f9d91a7c6c0 1 -- 10.20.193.27:0/4138477229 shutdown_connections 2026-02-13T02:22:20.030 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.027+0000 7f9d91a7c6c0 1 --2- 10.20.193.27:0/4138477229 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9d600b5680 0x7f9d600d5b30 unknown :-1 s=CLOSED pgs=169 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:20.031 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.027+0000 7f9d91a7c6c0 1 --2- 10.20.193.27:0/4138477229 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9d8c0f4c40 0x7f9d8c0f9f10 unknown :-1 s=CLOSED pgs=226 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:20.031 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.027+0000 7f9d91a7c6c0 1 --2- 10.20.193.27:0/4138477229 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9d8c1131d0 0x7f9d8c173a70 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:20.031 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.027+0000 7f9d91a7c6c0 1 --2- 10.20.193.27:0/4138477229 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9d8c10b360 0x7f9d8c0f4610 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:20.031 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.027+0000 7f9d91a7c6c0 1 -- 10.20.193.27:0/4138477229 >> 10.20.193.27:0/4138477229 conn(0x7f9d8c062e00 msgr2=0x7f9d8c111850 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:20.031 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.027+0000 7f9d91a7c6c0 1 -- 10.20.193.27:0/4138477229 shutdown_connections 2026-02-13T02:22:20.031 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.027+0000 7f9d91a7c6c0 1 -- 10.20.193.27:0/4138477229 wait complete. 2026-02-13T02:22:20.071 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph fs set cephfs session_timeout 300 2026-02-13T02:22:20.187 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:20.301 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.298+0000 7f8c7c7b06c0 1 -- 10.20.193.27:0/2900741197 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8c7405f810 msgr2=0x7f8c7405fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:20.301 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.298+0000 7f8c7c7b06c0 1 --2- 10.20.193.27:0/2900741197 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8c7405f810 0x7f8c7405fbe0 secure :-1 s=READY pgs=227 gs=1 cs=0 l=1 c_cookie=1303760848188972525 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8c640099a0 tx=0x7f8c6402ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:20.301 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.298+0000 7f8c7c7b06c0 1 -- 10.20.193.27:0/2900741197 shutdown_connections 2026-02-13T02:22:20.301 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.298+0000 7f8c7c7b06c0 1 --2- 10.20.193.27:0/2900741197 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8c741131b0 0x7f8c74173b50 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:20.301 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.298+0000 7f8c7c7b06c0 1 --2- 10.20.193.27:0/2900741197 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8c74060200 0x7f8c741129d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:20.301 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.298+0000 7f8c7c7b06c0 1 --2- 10.20.193.27:0/2900741197 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8c7405f810 0x7f8c7405fbe0 unknown :-1 s=CLOSED pgs=227 gs=1 cs=0 l=1 c_cookie=1303760848188972525 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:20.301 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.298+0000 7f8c7c7b06c0 1 -- 10.20.193.27:0/2900741197 >> 10.20.193.27:0/2900741197 conn(0x7f8c740e9740 msgr2=0x7f8c74109b50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:20.302 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.298+0000 7f8c7c7b06c0 1 -- 10.20.193.27:0/2900741197 shutdown_connections 2026-02-13T02:22:20.302 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.298+0000 7f8c7c7b06c0 1 -- 10.20.193.27:0/2900741197 wait complete. 2026-02-13T02:22:20.302 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:20 trial027 ceph-mon[24567]: pgmap v185: 97 pgs: 16 creating+peering, 49 unknown, 32 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail; 1.7 KiB/s wr, 2 op/s 2026-02-13T02:22:20.302 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:20 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1998777205' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2026-02-13T02:22:20.302 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:20 trial027 ceph-mon[24567]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2026-02-13T02:22:20.302 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:20 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/4138477229' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2026-02-13T02:22:20.305 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.298+0000 7f8c7c7b06c0 1 Processor -- start 2026-02-13T02:22:20.305 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.298+0000 7f8c7c7b06c0 1 -- start start 2026-02-13T02:22:20.305 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.299+0000 7f8c7c7b06c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8c7405f810 0x7f8c740f3be0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:20.305 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.299+0000 7f8c7c7b06c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8c741131b0 0x7f8c74173a50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:20.305 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.299+0000 7f8c7c7b06c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8c740f4210 0x7f8c740f94e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:20.305 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.299+0000 7f8c7c7b06c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8c740f4900 con 0x7f8c741131b0 2026-02-13T02:22:20.305 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.299+0000 7f8c7c7b06c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8c740f9a20 con 0x7f8c7405f810 2026-02-13T02:22:20.306 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.299+0000 7f8c7c7b06c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8c740f9be0 con 0x7f8c740f4210 2026-02-13T02:22:20.306 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.299+0000 7f8c79d246c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8c741131b0 0x7f8c74173a50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:20.306 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.299+0000 7f8c79d246c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8c741131b0 0x7f8c74173a50 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55246/0 (socket says 10.20.193.27:55246) 2026-02-13T02:22:20.306 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.299+0000 7f8c79d246c0 1 -- 10.20.193.27:0/3166151325 learned_addr learned my addr 10.20.193.27:0/3166151325 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:20.306 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.299+0000 7f8c7a5256c0 1 --2- 10.20.193.27:0/3166151325 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8c7405f810 0x7f8c740f3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:20.306 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.299+0000 7f8c7ad266c0 1 --2- 10.20.193.27:0/3166151325 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8c740f4210 0x7f8c740f94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:20.306 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.299+0000 7f8c79d246c0 1 -- 10.20.193.27:0/3166151325 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8c740f4210 msgr2=0x7f8c740f94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:20.306 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.299+0000 7f8c79d246c0 1 --2- 10.20.193.27:0/3166151325 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8c740f4210 0x7f8c740f94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:20.306 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.299+0000 7f8c79d246c0 1 -- 10.20.193.27:0/3166151325 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8c7405f810 msgr2=0x7f8c740f3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:20.306 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.299+0000 7f8c79d246c0 1 --2- 10.20.193.27:0/3166151325 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8c7405f810 0x7f8c740f3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:20.306 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.299+0000 7f8c79d246c0 1 -- 10.20.193.27:0/3166151325 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c64009590 con 0x7f8c741131b0 2026-02-13T02:22:20.306 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.299+0000 7f8c79d246c0 1 --2- 10.20.193.27:0/3166151325 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8c741131b0 0x7f8c74173a50 secure :-1 s=READY pgs=228 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8c68005dc0 tx=0x7f8c6800bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:20.307 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.299+0000 7f8c5b7fe6c0 1 -- 10.20.193.27:0/3166151325 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8c68004250 con 0x7f8c741131b0 2026-02-13T02:22:20.307 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.299+0000 7f8c5b7fe6c0 1 -- 10.20.193.27:0/3166151325 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f8c68009070 con 0x7f8c741131b0 2026-02-13T02:22:20.307 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.299+0000 7f8c7c7b06c0 1 -- 10.20.193.27:0/3166151325 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c740f9ec0 con 0x7f8c741131b0 2026-02-13T02:22:20.307 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.299+0000 7f8c5b7fe6c0 1 -- 10.20.193.27:0/3166151325 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8c6800cde0 con 0x7f8c741131b0 2026-02-13T02:22:20.307 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.299+0000 7f8c7c7b06c0 1 -- 10.20.193.27:0/3166151325 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c740fa650 con 0x7f8c741131b0 2026-02-13T02:22:20.307 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.300+0000 7f8c7c7b06c0 1 -- 10.20.193.27:0/3166151325 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c7410e270 con 0x7f8c741131b0 2026-02-13T02:22:20.307 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.301+0000 7f8c5b7fe6c0 1 -- 10.20.193.27:0/3166151325 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7f8c680028c0 con 0x7f8c741131b0 2026-02-13T02:22:20.307 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.301+0000 7f8c5b7fe6c0 1 --2- 10.20.193.27:0/3166151325 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8c500b5680 0x7f8c500d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:20.307 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.302+0000 7f8c5b7fe6c0 1 -- 10.20.193.27:0/3166151325 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f8c680e3810 con 0x7f8c741131b0 2026-02-13T02:22:20.307 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.302+0000 7f8c7a5256c0 1 --2- 10.20.193.27:0/3166151325 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8c500b5680 0x7f8c500d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:20.307 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.302+0000 7f8c5b7fe6c0 1 -- 10.20.193.27:0/3166151325 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f8c680a6390 con 0x7f8c741131b0 2026-02-13T02:22:20.307 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.302+0000 7f8c7a5256c0 1 --2- 10.20.193.27:0/3166151325 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8c500b5680 0x7f8c500d5b30 secure :-1 s=READY pgs=170 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8c64005b40 tx=0x7f8c640023a0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:20.417 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:20.415+0000 7f8c7c7b06c0 1 -- 10.20.193.27:0/3166151325 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f8c7405fbe0 con 0x7f8c741131b0 2026-02-13T02:22:20.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:20 trial028 ceph-mon[26701]: pgmap v185: 97 pgs: 16 creating+peering, 49 unknown, 32 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail; 1.7 KiB/s wr, 2 op/s 2026-02-13T02:22:20.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:20 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1998777205' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2026-02-13T02:22:20.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:20 trial028 ceph-mon[26701]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2026-02-13T02:22:20.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:20 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/4138477229' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2026-02-13T02:22:20.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:20 trial107 ceph-mon[27422]: pgmap v185: 97 pgs: 16 creating+peering, 49 unknown, 32 active+clean; 577 KiB data, 324 MiB used, 8.2 TiB / 8.2 TiB avail; 1.7 KiB/s wr, 2 op/s 2026-02-13T02:22:20.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:20 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1998777205' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2026-02-13T02:22:20.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:20 trial107 ceph-mon[27422]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2026-02-13T02:22:20.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:20 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/4138477229' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2026-02-13T02:22:21.035 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.033+0000 7f8c5b7fe6c0 1 -- 10.20.193.27:0/3166151325 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v18) ==== 115+0+0 (secure 0 0 0) 0x7f8c680a9540 con 0x7f8c741131b0 2026-02-13T02:22:21.037 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.034+0000 7f8c7c7b06c0 1 -- 10.20.193.27:0/3166151325 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8c500b5680 msgr2=0x7f8c500d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:21.037 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.034+0000 7f8c7c7b06c0 1 --2- 10.20.193.27:0/3166151325 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8c500b5680 0x7f8c500d5b30 secure :-1 s=READY pgs=170 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8c64005b40 tx=0x7f8c640023a0 comp rx=0 tx=0).stop 2026-02-13T02:22:21.038 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.034+0000 7f8c7c7b06c0 1 -- 10.20.193.27:0/3166151325 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8c741131b0 msgr2=0x7f8c74173a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:21.038 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.034+0000 7f8c7c7b06c0 1 --2- 10.20.193.27:0/3166151325 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8c741131b0 0x7f8c74173a50 secure :-1 s=READY pgs=228 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8c68005dc0 tx=0x7f8c6800bbb0 comp rx=0 tx=0).stop 2026-02-13T02:22:21.038 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.034+0000 7f8c7c7b06c0 1 -- 10.20.193.27:0/3166151325 shutdown_connections 2026-02-13T02:22:21.038 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.034+0000 7f8c7c7b06c0 1 --2- 10.20.193.27:0/3166151325 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8c500b5680 0x7f8c500d5b30 unknown :-1 s=CLOSED pgs=170 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:21.038 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.034+0000 7f8c7c7b06c0 1 --2- 10.20.193.27:0/3166151325 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8c740f4210 0x7f8c740f94e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:21.038 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.034+0000 7f8c7c7b06c0 1 --2- 10.20.193.27:0/3166151325 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8c741131b0 0x7f8c74173a50 unknown :-1 s=CLOSED pgs=228 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:21.038 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.034+0000 7f8c7c7b06c0 1 --2- 10.20.193.27:0/3166151325 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8c7405f810 0x7f8c740f3be0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:21.038 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.034+0000 7f8c7c7b06c0 1 -- 10.20.193.27:0/3166151325 >> 10.20.193.27:0/3166151325 conn(0x7f8c740e9740 msgr2=0x7f8c7410f950 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:21.038 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.035+0000 7f8c7c7b06c0 1 -- 10.20.193.27:0/3166151325 shutdown_connections 2026-02-13T02:22:21.038 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.035+0000 7f8c7c7b06c0 1 -- 10.20.193.27:0/3166151325 wait complete. 2026-02-13T02:22:21.078 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2026-02-13T02:22:21.078 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph fs subvolumegroup create cephfs qa 2026-02-13T02:22:21.196 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:21.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.305+0000 7f9adf7856c0 1 -- 10.20.193.27:0/1546549906 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9ad8115440 msgr2=0x7f9ad8175de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:21.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.305+0000 7f9adf7856c0 1 --2- 10.20.193.27:0/1546549906 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9ad8115440 0x7f9ad8175de0 secure :-1 s=READY pgs=229 gs=1 cs=0 l=1 c_cookie=13043425656887119587 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9ad400aa70 tx=0x7f9ad402fde0 comp rx=0 tx=0).stop 2026-02-13T02:22:21.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.305+0000 7f9adf7856c0 1 -- 10.20.193.27:0/1546549906 shutdown_connections 2026-02-13T02:22:21.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.305+0000 7f9adf7856c0 1 --2- 10.20.193.27:0/1546549906 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9ad8115440 0x7f9ad8175de0 unknown :-1 s=CLOSED pgs=229 gs=1 cs=0 l=1 c_cookie=13043425656887119587 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:21.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.305+0000 7f9adf7856c0 1 --2- 10.20.193.27:0/1546549906 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9ad810fc20 0x7f9ad8114c60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:21.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.305+0000 7f9adf7856c0 1 --2- 10.20.193.27:0/1546549906 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9ad810f230 0x7f9ad810f600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:21.309 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.305+0000 7f9adf7856c0 1 -- 10.20.193.27:0/1546549906 >> 10.20.193.27:0/1546549906 conn(0x7f9ad80ecb30 msgr2=0x7f9ad810cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:21.309 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.305+0000 7f9adf7856c0 1 -- 10.20.193.27:0/1546549906 shutdown_connections 2026-02-13T02:22:21.309 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.305+0000 7f9adf7856c0 1 -- 10.20.193.27:0/1546549906 wait complete. 2026-02-13T02:22:21.309 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:21 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/4138477229' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2026-02-13T02:22:21.309 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:21 trial027 ceph-mon[24567]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2026-02-13T02:22:21.309 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:21 trial027 ceph-mon[24567]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2026-02-13T02:22:21.309 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:21 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3166151325' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2026-02-13T02:22:21.312 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.305+0000 7f9adf7856c0 1 Processor -- start 2026-02-13T02:22:21.312 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.305+0000 7f9adf7856c0 1 -- start start 2026-02-13T02:22:21.312 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.306+0000 7f9adf7856c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9ad810f230 0x7f9ad811d1e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:21.312 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.306+0000 7f9adf7856c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9ad8115440 0x7f9ad811d720 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:21.312 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.306+0000 7f9adf7856c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9ad8122ae0 0x7f9ad8122f80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:21.313 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.306+0000 7f9adf7856c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad81234f0 con 0x7f9ad8122ae0 2026-02-13T02:22:21.313 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.306+0000 7f9adf7856c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad8123660 con 0x7f9ad8115440 2026-02-13T02:22:21.313 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.306+0000 7f9adf7856c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad8123820 con 0x7f9ad810f230 2026-02-13T02:22:21.313 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.306+0000 7f9addcf76c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9ad8122ae0 0x7f9ad8122f80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:21.313 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.306+0000 7f9addcf76c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9ad8122ae0 0x7f9ad8122f80 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55256/0 (socket says 10.20.193.27:55256) 2026-02-13T02:22:21.313 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.306+0000 7f9addcf76c0 1 -- 10.20.193.27:0/1079363312 learned_addr learned my addr 10.20.193.27:0/1079363312 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:21.313 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.306+0000 7f9adccf56c0 1 --2- 10.20.193.27:0/1079363312 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9ad8115440 0x7f9ad811d720 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:21.313 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.306+0000 7f9add4f66c0 1 --2- 10.20.193.27:0/1079363312 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9ad810f230 0x7f9ad811d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:21.313 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.306+0000 7f9addcf76c0 1 -- 10.20.193.27:0/1079363312 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9ad810f230 msgr2=0x7f9ad811d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:21.313 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.306+0000 7f9addcf76c0 1 --2- 10.20.193.27:0/1079363312 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9ad810f230 0x7f9ad811d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:21.313 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.306+0000 7f9addcf76c0 1 -- 10.20.193.27:0/1079363312 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9ad8115440 msgr2=0x7f9ad811d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:21.314 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.306+0000 7f9addcf76c0 1 --2- 10.20.193.27:0/1079363312 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9ad8115440 0x7f9ad811d720 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:21.314 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.306+0000 7f9addcf76c0 1 -- 10.20.193.27:0/1079363312 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad400a660 con 0x7f9ad8122ae0 2026-02-13T02:22:21.314 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.306+0000 7f9addcf76c0 1 --2- 10.20.193.27:0/1079363312 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9ad8122ae0 0x7f9ad8122f80 secure :-1 s=READY pgs=230 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9ad40054b0 tx=0x7f9ad40054e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:21.314 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.306+0000 7f9ac67fc6c0 1 -- 10.20.193.27:0/1079363312 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9ad4003a30 con 0x7f9ad8122ae0 2026-02-13T02:22:21.314 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.306+0000 7f9ac67fc6c0 1 -- 10.20.193.27:0/1079363312 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9ad4003b90 con 0x7f9ad8122ae0 2026-02-13T02:22:21.314 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.306+0000 7f9adf7856c0 1 -- 10.20.193.27:0/1079363312 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ad8163c70 con 0x7f9ad8122ae0 2026-02-13T02:22:21.314 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.306+0000 7f9ac67fc6c0 1 -- 10.20.193.27:0/1079363312 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9ad40382f0 con 0x7f9ad8122ae0 2026-02-13T02:22:21.314 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.306+0000 7f9adf7856c0 1 -- 10.20.193.27:0/1079363312 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ad81640a0 con 0x7f9ad8122ae0 2026-02-13T02:22:21.314 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.307+0000 7f9adf7856c0 1 -- 10.20.193.27:0/1079363312 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ad8110500 con 0x7f9ad8122ae0 2026-02-13T02:22:21.314 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.308+0000 7f9ac67fc6c0 1 -- 10.20.193.27:0/1079363312 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7f9ad4008260 con 0x7f9ad8122ae0 2026-02-13T02:22:21.314 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.308+0000 7f9ac67fc6c0 1 --2- 10.20.193.27:0/1079363312 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9ab40b5600 0x7f9ab40d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:21.314 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.309+0000 7f9add4f66c0 1 --2- 10.20.193.27:0/1079363312 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9ab40b5600 0x7f9ab40d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:21.315 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.309+0000 7f9ac67fc6c0 1 -- 10.20.193.27:0/1079363312 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f9ad4108d90 con 0x7f9ad8122ae0 2026-02-13T02:22:21.315 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.309+0000 7f9add4f66c0 1 --2- 10.20.193.27:0/1079363312 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9ab40b5600 0x7f9ab40d5ab0 secure :-1 s=READY pgs=171 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9ac8005980 tx=0x7f9ac8005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:21.315 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.309+0000 7f9ac67fc6c0 1 -- 10.20.193.27:0/1079363312 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f9ad40cb950 con 0x7f9ad8122ae0 2026-02-13T02:22:21.422 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.420+0000 7f9adf7856c0 1 -- 10.20.193.27:0/1079363312 --> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9ad811e3a0 con 0x7f9ab40b5600 2026-02-13T02:22:21.437 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.434+0000 7f9ac67fc6c0 1 -- 10.20.193.27:0/1079363312 <== mgr.14116 v2:10.20.193.27:6800/1837452956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9ad811e3a0 con 0x7f9ab40b5600 2026-02-13T02:22:21.439 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.436+0000 7f9adf7856c0 1 -- 10.20.193.27:0/1079363312 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9ab40b5600 msgr2=0x7f9ab40d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:21.439 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.436+0000 7f9adf7856c0 1 --2- 10.20.193.27:0/1079363312 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9ab40b5600 0x7f9ab40d5ab0 secure :-1 s=READY pgs=171 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9ac8005980 tx=0x7f9ac8005910 comp rx=0 tx=0).stop 2026-02-13T02:22:21.439 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.436+0000 7f9adf7856c0 1 -- 10.20.193.27:0/1079363312 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9ad8122ae0 msgr2=0x7f9ad8122f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:21.439 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.436+0000 7f9adf7856c0 1 --2- 10.20.193.27:0/1079363312 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9ad8122ae0 0x7f9ad8122f80 secure :-1 s=READY pgs=230 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9ad40054b0 tx=0x7f9ad40054e0 comp rx=0 tx=0).stop 2026-02-13T02:22:21.439 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.436+0000 7f9adf7856c0 1 -- 10.20.193.27:0/1079363312 shutdown_connections 2026-02-13T02:22:21.439 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.436+0000 7f9adf7856c0 1 --2- 10.20.193.27:0/1079363312 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9ab40b5600 0x7f9ab40d5ab0 unknown :-1 s=CLOSED pgs=171 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:21.439 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.436+0000 7f9adf7856c0 1 --2- 10.20.193.27:0/1079363312 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9ad8122ae0 0x7f9ad8122f80 unknown :-1 s=CLOSED pgs=230 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:21.439 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.436+0000 7f9adf7856c0 1 --2- 10.20.193.27:0/1079363312 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9ad8115440 0x7f9ad811d720 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:21.439 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.436+0000 7f9adf7856c0 1 --2- 10.20.193.27:0/1079363312 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9ad810f230 0x7f9ad811d1e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:21.439 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.436+0000 7f9adf7856c0 1 -- 10.20.193.27:0/1079363312 >> 10.20.193.27:0/1079363312 conn(0x7f9ad80ecb30 msgr2=0x7f9ad810cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:21.440 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.436+0000 7f9adf7856c0 1 -- 10.20.193.27:0/1079363312 shutdown_connections 2026-02-13T02:22:21.440 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.436+0000 7f9adf7856c0 1 -- 10.20.193.27:0/1079363312 wait complete. 2026-02-13T02:22:21.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:21 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/4138477229' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2026-02-13T02:22:21.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:21 trial028 ceph-mon[26701]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2026-02-13T02:22:21.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:21 trial028 ceph-mon[26701]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2026-02-13T02:22:21.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:21 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3166151325' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2026-02-13T02:22:21.495 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2026-02-13T02:22:21.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:21 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/4138477229' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2026-02-13T02:22:21.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:21 trial107 ceph-mon[27422]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2026-02-13T02:22:21.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:21 trial107 ceph-mon[27422]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2026-02-13T02:22:21.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:21 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3166151325' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2026-02-13T02:22:21.615 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:21.640 INFO:journalctl@ceph.mgr.x.trial027.stdout:Feb 13 02:22:21 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mgr-x[24825]: 2026-02-13T02:22:21.426+0000 7f94360cf6c0 -1 client.14817 error registering admin socket command: (17) File exists 2026-02-13T02:22:21.739 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.736+0000 7f6293a416c0 1 -- 10.20.193.27:0/2189868166 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f628c10f200 msgr2=0x7f628c10f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:21.739 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.736+0000 7f6293a416c0 1 --2- 10.20.193.27:0/2189868166 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f628c10f200 0x7f628c10f5d0 secure :-1 s=READY pgs=233 gs=1 cs=0 l=1 c_cookie=9991760843835656676 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f627c0099a0 tx=0x7f627c02ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:21.740 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.736+0000 7f6293a416c0 1 -- 10.20.193.27:0/2189868166 shutdown_connections 2026-02-13T02:22:21.740 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.736+0000 7f6293a416c0 1 --2- 10.20.193.27:0/2189868166 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f628c115410 0x7f628c175db0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:21.740 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.736+0000 7f6293a416c0 1 --2- 10.20.193.27:0/2189868166 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f628c10fbf0 0x7f628c114c30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:21.740 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.736+0000 7f6293a416c0 1 --2- 10.20.193.27:0/2189868166 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f628c10f200 0x7f628c10f5d0 unknown :-1 s=CLOSED pgs=233 gs=1 cs=0 l=1 c_cookie=9991760843835656676 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:21.740 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.736+0000 7f6293a416c0 1 -- 10.20.193.27:0/2189868166 >> 10.20.193.27:0/2189868166 conn(0x7f628c0ecb00 msgr2=0x7f628c10cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:21.740 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.736+0000 7f6293a416c0 1 -- 10.20.193.27:0/2189868166 shutdown_connections 2026-02-13T02:22:21.740 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.736+0000 7f6293a416c0 1 -- 10.20.193.27:0/2189868166 wait complete. 2026-02-13T02:22:21.740 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.736+0000 7f6293a416c0 1 Processor -- start 2026-02-13T02:22:21.740 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.736+0000 7f6293a416c0 1 -- start start 2026-02-13T02:22:21.740 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.736+0000 7f6293a416c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f628c10f200 0x7f628c11d260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:21.741 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.736+0000 7f6293a416c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f628c115410 0x7f628c11d7a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:21.741 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.736+0000 7f6293a416c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f628c122b60 0x7f628c123000 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:21.741 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.736+0000 7f6293a416c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f628c123540 con 0x7f628c115410 2026-02-13T02:22:21.741 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.736+0000 7f6293a416c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f628c1236b0 con 0x7f628c122b60 2026-02-13T02:22:21.741 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.736+0000 7f6293a416c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f628c123870 con 0x7f628c10f200 2026-02-13T02:22:21.741 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.736+0000 7f6290fb16c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f628c115410 0x7f628c11d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:21.741 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.736+0000 7f6290fb16c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f628c115410 0x7f628c11d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55290/0 (socket says 10.20.193.27:55290) 2026-02-13T02:22:21.741 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.736+0000 7f6290fb16c0 1 -- 10.20.193.27:0/1864575316 learned_addr learned my addr 10.20.193.27:0/1864575316 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:21.741 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.737+0000 7f62917b26c0 1 --2- 10.20.193.27:0/1864575316 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f628c10f200 0x7f628c11d260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:21.741 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.737+0000 7f6291fb36c0 1 --2- 10.20.193.27:0/1864575316 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f628c122b60 0x7f628c123000 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:21.741 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.737+0000 7f6290fb16c0 1 -- 10.20.193.27:0/1864575316 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f628c10f200 msgr2=0x7f628c11d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:21.741 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.737+0000 7f6290fb16c0 1 --2- 10.20.193.27:0/1864575316 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f628c10f200 0x7f628c11d260 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:21.742 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.737+0000 7f6290fb16c0 1 -- 10.20.193.27:0/1864575316 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f628c122b60 msgr2=0x7f628c123000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:21.742 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.737+0000 7f6290fb16c0 1 --2- 10.20.193.27:0/1864575316 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f628c122b60 0x7f628c123000 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:21.742 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.737+0000 7f6290fb16c0 1 -- 10.20.193.27:0/1864575316 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f627c009590 con 0x7f628c115410 2026-02-13T02:22:21.742 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.737+0000 7f6290fb16c0 1 --2- 10.20.193.27:0/1864575316 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f628c115410 0x7f628c11d7a0 secure :-1 s=READY pgs=234 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6280005dc0 tx=0x7f628000bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:21.742 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.737+0000 7f627a7fc6c0 1 -- 10.20.193.27:0/1864575316 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6280004250 con 0x7f628c115410 2026-02-13T02:22:21.742 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.737+0000 7f6293a416c0 1 -- 10.20.193.27:0/1864575316 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f628c163cf0 con 0x7f628c115410 2026-02-13T02:22:21.742 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.737+0000 7f627a7fc6c0 1 -- 10.20.193.27:0/1864575316 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6280009070 con 0x7f628c115410 2026-02-13T02:22:21.742 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.737+0000 7f627a7fc6c0 1 -- 10.20.193.27:0/1864575316 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f628000cd50 con 0x7f628c115410 2026-02-13T02:22:21.742 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.737+0000 7f6293a416c0 1 -- 10.20.193.27:0/1864575316 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f628c1641d0 con 0x7f628c115410 2026-02-13T02:22:21.742 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.737+0000 7f6293a416c0 1 -- 10.20.193.27:0/1864575316 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f628c1104d0 con 0x7f628c115410 2026-02-13T02:22:21.743 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.739+0000 7f627a7fc6c0 1 -- 10.20.193.27:0/1864575316 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7f62800028c0 con 0x7f628c115410 2026-02-13T02:22:21.743 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.739+0000 7f627a7fc6c0 1 --2- 10.20.193.27:0/1864575316 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f62680b5600 0x7f62680d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:21.744 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.739+0000 7f627a7fc6c0 1 -- 10.20.193.27:0/1864575316 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f62800e3990 con 0x7f628c115410 2026-02-13T02:22:21.744 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.739+0000 7f62917b26c0 1 --2- 10.20.193.27:0/1864575316 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f62680b5600 0x7f62680d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:21.744 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.739+0000 7f627a7fc6c0 1 -- 10.20.193.27:0/1864575316 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f62800a6550 con 0x7f628c115410 2026-02-13T02:22:21.744 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.740+0000 7f62917b26c0 1 --2- 10.20.193.27:0/1864575316 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f62680b5600 0x7f62680d5ab0 secure :-1 s=READY pgs=172 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f627c0023e0 tx=0x7f627c005de0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:21.849 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.848+0000 7f6293a416c0 1 -- 10.20.193.27:0/1864575316 --> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f628c11e440 con 0x7f62680b5600 2026-02-13T02:22:21.863 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.861+0000 7f627a7fc6c0 1 -- 10.20.193.27:0/1864575316 <== mgr.14116 v2:10.20.193.27:6800/1837452956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f627c004340 con 0x7f62680b5600 2026-02-13T02:22:21.864 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.863+0000 7f6293a416c0 1 -- 10.20.193.27:0/1864575316 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f62680b5600 msgr2=0x7f62680d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:21.865 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.863+0000 7f6293a416c0 1 --2- 10.20.193.27:0/1864575316 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f62680b5600 0x7f62680d5ab0 secure :-1 s=READY pgs=172 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f627c0023e0 tx=0x7f627c005de0 comp rx=0 tx=0).stop 2026-02-13T02:22:21.865 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.863+0000 7f6293a416c0 1 -- 10.20.193.27:0/1864575316 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f628c115410 msgr2=0x7f628c11d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:21.865 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.863+0000 7f6293a416c0 1 --2- 10.20.193.27:0/1864575316 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f628c115410 0x7f628c11d7a0 secure :-1 s=READY pgs=234 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6280005dc0 tx=0x7f628000bbb0 comp rx=0 tx=0).stop 2026-02-13T02:22:21.865 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.863+0000 7f6293a416c0 1 -- 10.20.193.27:0/1864575316 shutdown_connections 2026-02-13T02:22:21.865 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.863+0000 7f6293a416c0 1 --2- 10.20.193.27:0/1864575316 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f62680b5600 0x7f62680d5ab0 unknown :-1 s=CLOSED pgs=172 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:21.865 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.863+0000 7f6293a416c0 1 --2- 10.20.193.27:0/1864575316 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f628c122b60 0x7f628c123000 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:21.865 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.863+0000 7f6293a416c0 1 --2- 10.20.193.27:0/1864575316 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f628c115410 0x7f628c11d7a0 unknown :-1 s=CLOSED pgs=234 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:21.865 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.863+0000 7f6293a416c0 1 --2- 10.20.193.27:0/1864575316 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f628c10f200 0x7f628c11d260 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:21.865 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.863+0000 7f6293a416c0 1 -- 10.20.193.27:0/1864575316 >> 10.20.193.27:0/1864575316 conn(0x7f628c0ecb00 msgr2=0x7f628c10cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:21.865 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.863+0000 7f6293a416c0 1 -- 10.20.193.27:0/1864575316 shutdown_connections 2026-02-13T02:22:21.865 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:21.863+0000 7f6293a416c0 1 -- 10.20.193.27:0/1864575316 wait complete. 2026-02-13T02:22:21.904 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2026-02-13T02:22:22.023 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:22.148 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.144+0000 7ff69e7556c0 1 -- 10.20.193.27:0/770646039 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff69810f230 msgr2=0x7ff69810f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:22.148 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.144+0000 7ff69e7556c0 1 --2- 10.20.193.27:0/770646039 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff69810f230 0x7ff69810f600 secure :-1 s=READY pgs=235 gs=1 cs=0 l=1 c_cookie=11208822975530821938 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff6840099a0 tx=0x7ff68402ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:22.148 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.144+0000 7ff69e7556c0 1 -- 10.20.193.27:0/770646039 shutdown_connections 2026-02-13T02:22:22.148 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.144+0000 7ff69e7556c0 1 --2- 10.20.193.27:0/770646039 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff698115440 0x7ff698175de0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:22.148 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.144+0000 7ff69e7556c0 1 --2- 10.20.193.27:0/770646039 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff69810fc20 0x7ff698114c60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:22.148 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.144+0000 7ff69e7556c0 1 --2- 10.20.193.27:0/770646039 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff69810f230 0x7ff69810f600 unknown :-1 s=CLOSED pgs=235 gs=1 cs=0 l=1 c_cookie=11208822975530821938 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:22.148 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.144+0000 7ff69e7556c0 1 -- 10.20.193.27:0/770646039 >> 10.20.193.27:0/770646039 conn(0x7ff6980ecb30 msgr2=0x7ff69810cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:22.149 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.144+0000 7ff69e7556c0 1 -- 10.20.193.27:0/770646039 shutdown_connections 2026-02-13T02:22:22.149 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.144+0000 7ff69e7556c0 1 -- 10.20.193.27:0/770646039 wait complete. 2026-02-13T02:22:22.149 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.145+0000 7ff69e7556c0 1 Processor -- start 2026-02-13T02:22:22.149 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.145+0000 7ff69e7556c0 1 -- start start 2026-02-13T02:22:22.149 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.145+0000 7ff69e7556c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff69810f230 0x7ff69811d360 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:22.149 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.145+0000 7ff69e7556c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff698115440 0x7ff69811d8a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:22.149 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.145+0000 7ff69e7556c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff698122c60 0x7ff698123100 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:22.149 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.145+0000 7ff69e7556c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff698123670 con 0x7ff698122c60 2026-02-13T02:22:22.149 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.145+0000 7ff69e7556c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff6981237e0 con 0x7ff69810f230 2026-02-13T02:22:22.149 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.145+0000 7ff69e7556c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff6981239a0 con 0x7ff698115440 2026-02-13T02:22:22.149 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.145+0000 7ff69ccc76c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff698122c60 0x7ff698123100 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:22.150 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.145+0000 7ff69ccc76c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff698122c60 0x7ff698123100 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55314/0 (socket says 10.20.193.27:55314) 2026-02-13T02:22:22.150 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.145+0000 7ff69ccc76c0 1 -- 10.20.193.27:0/135912551 learned_addr learned my addr 10.20.193.27:0/135912551 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:22.150 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.145+0000 7ff697fff6c0 1 --2- 10.20.193.27:0/135912551 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff69810f230 0x7ff69811d360 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:22.150 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.145+0000 7ff6977fe6c0 1 --2- 10.20.193.27:0/135912551 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff698115440 0x7ff69811d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:22.150 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.145+0000 7ff69ccc76c0 1 -- 10.20.193.27:0/135912551 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff698115440 msgr2=0x7ff69811d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:22.150 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.145+0000 7ff69ccc76c0 1 --2- 10.20.193.27:0/135912551 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff698115440 0x7ff69811d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:22.150 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.145+0000 7ff69ccc76c0 1 -- 10.20.193.27:0/135912551 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff69810f230 msgr2=0x7ff69811d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:22.150 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.145+0000 7ff69ccc76c0 1 --2- 10.20.193.27:0/135912551 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff69810f230 0x7ff69811d360 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:22.150 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.145+0000 7ff69ccc76c0 1 -- 10.20.193.27:0/135912551 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff684009590 con 0x7ff698122c60 2026-02-13T02:22:22.150 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.145+0000 7ff69ccc76c0 1 --2- 10.20.193.27:0/135912551 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff698122c60 0x7ff698123100 secure :-1 s=READY pgs=236 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff68c006ea0 tx=0x7ff68c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:22.150 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.145+0000 7ff6957fa6c0 1 -- 10.20.193.27:0/135912551 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff68c003a30 con 0x7ff698122c60 2026-02-13T02:22:22.151 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.145+0000 7ff6957fa6c0 1 -- 10.20.193.27:0/135912551 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff68c0054f0 con 0x7ff698122c60 2026-02-13T02:22:22.151 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.145+0000 7ff69e7556c0 1 -- 10.20.193.27:0/135912551 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff698163e50 con 0x7ff698122c60 2026-02-13T02:22:22.151 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.145+0000 7ff6957fa6c0 1 -- 10.20.193.27:0/135912551 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff68c00dcd0 con 0x7ff698122c60 2026-02-13T02:22:22.151 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.145+0000 7ff69e7556c0 1 -- 10.20.193.27:0/135912551 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6981642e0 con 0x7ff698122c60 2026-02-13T02:22:22.151 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.146+0000 7ff69e7556c0 1 -- 10.20.193.27:0/135912551 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff698110500 con 0x7ff698122c60 2026-02-13T02:22:22.152 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.147+0000 7ff6957fa6c0 1 -- 10.20.193.27:0/135912551 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7ff68c005030 con 0x7ff698122c60 2026-02-13T02:22:22.152 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.148+0000 7ff6957fa6c0 1 --2- 10.20.193.27:0/135912551 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff66c0b5680 0x7ff66c0d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:22.152 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.148+0000 7ff697fff6c0 1 --2- 10.20.193.27:0/135912551 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff66c0b5680 0x7ff66c0d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:22.152 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.148+0000 7ff6957fa6c0 1 -- 10.20.193.27:0/135912551 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7ff68c0e4940 con 0x7ff698122c60 2026-02-13T02:22:22.152 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.148+0000 7ff697fff6c0 1 --2- 10.20.193.27:0/135912551 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff66c0b5680 0x7ff66c0d5b30 secure :-1 s=READY pgs=173 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff684005b40 tx=0x7ff6840023a0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:22.152 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.148+0000 7ff6957fa6c0 1 -- 10.20.193.27:0/135912551 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7ff68c0aaf20 con 0x7ff698122c60 2026-02-13T02:22:22.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:22 trial027 ceph-mon[24567]: pgmap v186: 97 pgs: 14 creating+peering, 83 active+clean; 577 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.7 KiB/s wr, 3 op/s 2026-02-13T02:22:22.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:22 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3166151325' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2026-02-13T02:22:22.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:22 trial027 ceph-mon[24567]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2026-02-13T02:22:22.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:22 trial027 ceph-mon[24567]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:22:22.243 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:22 trial028 ceph-mon[26701]: pgmap v186: 97 pgs: 14 creating+peering, 83 active+clean; 577 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.7 KiB/s wr, 3 op/s 2026-02-13T02:22:22.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:22 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3166151325' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2026-02-13T02:22:22.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:22 trial028 ceph-mon[26701]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2026-02-13T02:22:22.244 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:22 trial028 ceph-mon[26701]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:22:22.244 INFO:journalctl@ceph.mds.b.trial028.stdout:Feb 13 02:22:21 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-b[54870]: 2026-02-13T02:22:21.795+0000 7f565a2616c0 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2026-02-13T02:22:22.258 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.257+0000 7ff69e7556c0 1 -- 10.20.193.27:0/135912551 --> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7ff69811e540 con 0x7ff66c0b5680 2026-02-13T02:22:22.261 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.260+0000 7ff6957fa6c0 1 -- 10.20.193.27:0/135912551 <== mgr.14116 v2:10.20.193.27:6800/1837452956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7ff684004340 con 0x7ff66c0b5680 2026-02-13T02:22:22.262 INFO:teuthology.orchestra.run.trial027.stdout:/volumes/qa/sv_0/5cdb56b1-e380-40c7-8d38-39a2c8b37724 2026-02-13T02:22:22.263 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.261+0000 7ff69e7556c0 1 -- 10.20.193.27:0/135912551 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff66c0b5680 msgr2=0x7ff66c0d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:22.263 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.261+0000 7ff69e7556c0 1 --2- 10.20.193.27:0/135912551 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff66c0b5680 0x7ff66c0d5b30 secure :-1 s=READY pgs=173 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff684005b40 tx=0x7ff6840023a0 comp rx=0 tx=0).stop 2026-02-13T02:22:22.263 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.261+0000 7ff69e7556c0 1 -- 10.20.193.27:0/135912551 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff698122c60 msgr2=0x7ff698123100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:22.263 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.261+0000 7ff69e7556c0 1 --2- 10.20.193.27:0/135912551 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff698122c60 0x7ff698123100 secure :-1 s=READY pgs=236 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff68c006ea0 tx=0x7ff68c00cca0 comp rx=0 tx=0).stop 2026-02-13T02:22:22.263 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.261+0000 7ff69e7556c0 1 -- 10.20.193.27:0/135912551 shutdown_connections 2026-02-13T02:22:22.263 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.261+0000 7ff69e7556c0 1 --2- 10.20.193.27:0/135912551 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff66c0b5680 0x7ff66c0d5b30 unknown :-1 s=CLOSED pgs=173 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:22.263 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.261+0000 7ff69e7556c0 1 --2- 10.20.193.27:0/135912551 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff698122c60 0x7ff698123100 unknown :-1 s=CLOSED pgs=236 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:22.263 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.261+0000 7ff69e7556c0 1 --2- 10.20.193.27:0/135912551 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff698115440 0x7ff69811d8a0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:22.263 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.261+0000 7ff69e7556c0 1 --2- 10.20.193.27:0/135912551 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff69810f230 0x7ff69811d360 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:22.264 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.261+0000 7ff69e7556c0 1 -- 10.20.193.27:0/135912551 >> 10.20.193.27:0/135912551 conn(0x7ff6980ecb30 msgr2=0x7ff69810cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:22.264 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.261+0000 7ff69e7556c0 1 -- 10.20.193.27:0/135912551 shutdown_connections 2026-02-13T02:22:22.264 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.261+0000 7ff69e7556c0 1 -- 10.20.193.27:0/135912551 wait complete. 2026-02-13T02:22:22.303 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2026-02-13T02:22:22.418 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:22.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:22 trial107 ceph-mon[27422]: pgmap v186: 97 pgs: 14 creating+peering, 83 active+clean; 577 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.7 KiB/s wr, 3 op/s 2026-02-13T02:22:22.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:22 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3166151325' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2026-02-13T02:22:22.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:22 trial107 ceph-mon[27422]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2026-02-13T02:22:22.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:22 trial107 ceph-mon[27422]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:22:22.533 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.529+0000 7f84ec5d06c0 1 -- 10.20.193.27:0/3769123622 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f84e410fbf0 msgr2=0x7f84e4114c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:22.533 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.529+0000 7f84ec5d06c0 1 --2- 10.20.193.27:0/3769123622 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f84e410fbf0 0x7f84e4114c30 secure :-1 s=READY pgs=237 gs=1 cs=0 l=1 c_cookie=16863677218478616899 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f84d80099a0 tx=0x7f84d802ed50 comp rx=0 tx=0).stop 2026-02-13T02:22:22.534 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.529+0000 7f84ec5d06c0 1 -- 10.20.193.27:0/3769123622 shutdown_connections 2026-02-13T02:22:22.534 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.529+0000 7f84ec5d06c0 1 --2- 10.20.193.27:0/3769123622 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f84e4115410 0x7f84e4175db0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:22.534 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.529+0000 7f84ec5d06c0 1 --2- 10.20.193.27:0/3769123622 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f84e410fbf0 0x7f84e4114c30 unknown :-1 s=CLOSED pgs=237 gs=1 cs=0 l=1 c_cookie=16863677218478616899 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:22.534 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.529+0000 7f84ec5d06c0 1 --2- 10.20.193.27:0/3769123622 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f84e410f200 0x7f84e410f5d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:22.534 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.529+0000 7f84ec5d06c0 1 -- 10.20.193.27:0/3769123622 >> 10.20.193.27:0/3769123622 conn(0x7f84e40ecb00 msgr2=0x7f84e410cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:22.534 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.529+0000 7f84ec5d06c0 1 -- 10.20.193.27:0/3769123622 shutdown_connections 2026-02-13T02:22:22.534 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.529+0000 7f84ec5d06c0 1 -- 10.20.193.27:0/3769123622 wait complete. 2026-02-13T02:22:22.534 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.530+0000 7f84ec5d06c0 1 Processor -- start 2026-02-13T02:22:22.534 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.530+0000 7f84ec5d06c0 1 -- start start 2026-02-13T02:22:22.534 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.530+0000 7f84ec5d06c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f84e410f200 0x7f84e40f4ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:22.534 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.530+0000 7f84ec5d06c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f84e4115410 0x7f84e4175cb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:22.535 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.530+0000 7f84ec5d06c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f84e40f5310 0x7f84e40fa5e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:22.535 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.530+0000 7f84ec5d06c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f84e40f5a00 con 0x7f84e410f200 2026-02-13T02:22:22.535 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.530+0000 7f84ec5d06c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f84e40fab20 con 0x7f84e4115410 2026-02-13T02:22:22.535 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.530+0000 7f84ec5d06c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f84e40face0 con 0x7f84e40f5310 2026-02-13T02:22:22.535 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.530+0000 7f84ea3416c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f84e410f200 0x7f84e40f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:22.535 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.530+0000 7f84ea3416c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f84e410f200 0x7f84e40f4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55334/0 (socket says 10.20.193.27:55334) 2026-02-13T02:22:22.535 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.530+0000 7f84ea3416c0 1 -- 10.20.193.27:0/1132592892 learned_addr learned my addr 10.20.193.27:0/1132592892 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:22.535 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.530+0000 7f84e9b406c0 1 --2- 10.20.193.27:0/1132592892 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f84e4115410 0x7f84e4175cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:22.535 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.530+0000 7f84ea3416c0 1 -- 10.20.193.27:0/1132592892 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f84e40f5310 msgr2=0x7f84e40fa5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:22.535 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.530+0000 7f84eab426c0 1 --2- 10.20.193.27:0/1132592892 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f84e40f5310 0x7f84e40fa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:22.535 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.530+0000 7f84ea3416c0 1 --2- 10.20.193.27:0/1132592892 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f84e40f5310 0x7f84e40fa5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:22.536 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.530+0000 7f84ea3416c0 1 -- 10.20.193.27:0/1132592892 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f84e4115410 msgr2=0x7f84e4175cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:22.536 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.530+0000 7f84ea3416c0 1 --2- 10.20.193.27:0/1132592892 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f84e4115410 0x7f84e4175cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:22.536 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.530+0000 7f84ea3416c0 1 -- 10.20.193.27:0/1132592892 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d8009590 con 0x7f84e410f200 2026-02-13T02:22:22.536 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.530+0000 7f84e9b406c0 1 --2- 10.20.193.27:0/1132592892 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f84e4115410 0x7f84e4175cb0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:22:22.536 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.530+0000 7f84eab426c0 1 --2- 10.20.193.27:0/1132592892 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f84e40f5310 0x7f84e40fa5e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:22:22.536 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.530+0000 7f84ea3416c0 1 --2- 10.20.193.27:0/1132592892 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f84e410f200 0x7f84e40f4ce0 secure :-1 s=READY pgs=238 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f84d400b710 tx=0x7f84d400bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:22.536 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.530+0000 7f84cb7fe6c0 1 -- 10.20.193.27:0/1132592892 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f84d4004250 con 0x7f84e410f200 2026-02-13T02:22:22.536 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.530+0000 7f84cb7fe6c0 1 -- 10.20.193.27:0/1132592892 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f84d4002d80 con 0x7f84e410f200 2026-02-13T02:22:22.536 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.530+0000 7f84ec5d06c0 1 -- 10.20.193.27:0/1132592892 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e40fafc0 con 0x7f84e410f200 2026-02-13T02:22:22.536 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.530+0000 7f84cb7fe6c0 1 -- 10.20.193.27:0/1132592892 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f84d400ce50 con 0x7f84e410f200 2026-02-13T02:22:22.536 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.530+0000 7f84ec5d06c0 1 -- 10.20.193.27:0/1132592892 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e40fb750 con 0x7f84e410f200 2026-02-13T02:22:22.536 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.531+0000 7f84ec5d06c0 1 -- 10.20.193.27:0/1132592892 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84e41104d0 con 0x7f84e410f200 2026-02-13T02:22:22.537 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.532+0000 7f84cb7fe6c0 1 -- 10.20.193.27:0/1132592892 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7f84d40043b0 con 0x7f84e410f200 2026-02-13T02:22:22.537 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.533+0000 7f84cb7fe6c0 1 --2- 10.20.193.27:0/1132592892 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f84c00b5670 0x7f84c00d5b20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:22.537 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.533+0000 7f84e9b406c0 1 --2- 10.20.193.27:0/1132592892 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f84c00b5670 0x7f84c00d5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:22.538 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.533+0000 7f84cb7fe6c0 1 -- 10.20.193.27:0/1132592892 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f84d40e3940 con 0x7f84e410f200 2026-02-13T02:22:22.538 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.533+0000 7f84e9b406c0 1 --2- 10.20.193.27:0/1132592892 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f84c00b5670 0x7f84c00d5b20 secure :-1 s=READY pgs=174 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f84d80023e0 tx=0x7f84d80091e0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:22.538 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.533+0000 7f84cb7fe6c0 1 -- 10.20.193.27:0/1132592892 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f84d40a94d0 con 0x7f84e410f200 2026-02-13T02:22:22.645 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.644+0000 7f84ec5d06c0 1 -- 10.20.193.27:0/1132592892 --> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f84e40f5e30 con 0x7f84c00b5670 2026-02-13T02:22:22.659 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.658+0000 7f84cb7fe6c0 1 -- 10.20.193.27:0/1132592892 <== mgr.14116 v2:10.20.193.27:6800/1837452956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f84e40f5e30 con 0x7f84c00b5670 2026-02-13T02:22:22.660 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.659+0000 7f84ec5d06c0 1 -- 10.20.193.27:0/1132592892 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f84c00b5670 msgr2=0x7f84c00d5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:22.661 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.659+0000 7f84ec5d06c0 1 --2- 10.20.193.27:0/1132592892 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f84c00b5670 0x7f84c00d5b20 secure :-1 s=READY pgs=174 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f84d80023e0 tx=0x7f84d80091e0 comp rx=0 tx=0).stop 2026-02-13T02:22:22.661 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.659+0000 7f84ec5d06c0 1 -- 10.20.193.27:0/1132592892 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f84e410f200 msgr2=0x7f84e40f4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:22.661 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.659+0000 7f84ec5d06c0 1 --2- 10.20.193.27:0/1132592892 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f84e410f200 0x7f84e40f4ce0 secure :-1 s=READY pgs=238 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f84d400b710 tx=0x7f84d400bbe0 comp rx=0 tx=0).stop 2026-02-13T02:22:22.661 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.659+0000 7f84ec5d06c0 1 -- 10.20.193.27:0/1132592892 shutdown_connections 2026-02-13T02:22:22.661 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.659+0000 7f84ec5d06c0 1 --2- 10.20.193.27:0/1132592892 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f84c00b5670 0x7f84c00d5b20 unknown :-1 s=CLOSED pgs=174 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:22.661 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.659+0000 7f84ec5d06c0 1 --2- 10.20.193.27:0/1132592892 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f84e40f5310 0x7f84e40fa5e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:22.661 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.659+0000 7f84ec5d06c0 1 --2- 10.20.193.27:0/1132592892 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f84e4115410 0x7f84e4175cb0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:22.661 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.659+0000 7f84ec5d06c0 1 --2- 10.20.193.27:0/1132592892 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f84e410f200 0x7f84e40f4ce0 unknown :-1 s=CLOSED pgs=238 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:22.661 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.659+0000 7f84ec5d06c0 1 -- 10.20.193.27:0/1132592892 >> 10.20.193.27:0/1132592892 conn(0x7f84e40ecb00 msgr2=0x7f84e410d900 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:22.661 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.659+0000 7f84ec5d06c0 1 -- 10.20.193.27:0/1132592892 shutdown_connections 2026-02-13T02:22:22.662 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.659+0000 7f84ec5d06c0 1 -- 10.20.193.27:0/1132592892 wait complete. 2026-02-13T02:22:22.701 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2026-02-13T02:22:22.818 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:22.933 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.930+0000 7fc1d22946c0 1 -- 10.20.193.27:0/2877562512 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc1cc115410 msgr2=0x7fc1cc175db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:22.934 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.930+0000 7fc1d22946c0 1 --2- 10.20.193.27:0/2877562512 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc1cc115410 0x7fc1cc175db0 secure :-1 s=READY pgs=239 gs=1 cs=0 l=1 c_cookie=12864473157774900728 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc1c000aa70 tx=0x7fc1c002fde0 comp rx=0 tx=0).stop 2026-02-13T02:22:22.934 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.930+0000 7fc1d22946c0 1 -- 10.20.193.27:0/2877562512 shutdown_connections 2026-02-13T02:22:22.934 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.930+0000 7fc1d22946c0 1 --2- 10.20.193.27:0/2877562512 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc1cc115410 0x7fc1cc175db0 unknown :-1 s=CLOSED pgs=239 gs=1 cs=0 l=1 c_cookie=12864473157774900728 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:22.934 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.930+0000 7fc1d22946c0 1 --2- 10.20.193.27:0/2877562512 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc1cc10fbf0 0x7fc1cc114c30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:22.934 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.930+0000 7fc1d22946c0 1 --2- 10.20.193.27:0/2877562512 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc1cc10f200 0x7fc1cc10f5d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:22.934 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.930+0000 7fc1d22946c0 1 -- 10.20.193.27:0/2877562512 >> 10.20.193.27:0/2877562512 conn(0x7fc1cc0ecb00 msgr2=0x7fc1cc10cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:22.934 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.930+0000 7fc1d22946c0 1 -- 10.20.193.27:0/2877562512 shutdown_connections 2026-02-13T02:22:22.934 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.930+0000 7fc1d22946c0 1 -- 10.20.193.27:0/2877562512 wait complete. 2026-02-13T02:22:22.935 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.930+0000 7fc1d22946c0 1 Processor -- start 2026-02-13T02:22:22.935 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.930+0000 7fc1d22946c0 1 -- start start 2026-02-13T02:22:22.935 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.931+0000 7fc1d22946c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc1cc10f200 0x7fc1cc11d2b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:22.935 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.931+0000 7fc1d22946c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc1cc115410 0x7fc1cc11d7f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:22.935 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.931+0000 7fc1d22946c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc1cc122bb0 0x7fc1cc123050 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:22.935 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.931+0000 7fc1d22946c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc1235c0 con 0x7fc1cc10f200 2026-02-13T02:22:22.935 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.931+0000 7fc1d22946c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc123730 con 0x7fc1cc122bb0 2026-02-13T02:22:22.935 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.931+0000 7fc1d22946c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc1238f0 con 0x7fc1cc115410 2026-02-13T02:22:22.935 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.931+0000 7fc1cbfff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc1cc10f200 0x7fc1cc11d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:22.935 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.931+0000 7fc1cbfff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc1cc10f200 0x7fc1cc11d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55354/0 (socket says 10.20.193.27:55354) 2026-02-13T02:22:22.935 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.931+0000 7fc1cbfff6c0 1 -- 10.20.193.27:0/1560945546 learned_addr learned my addr 10.20.193.27:0/1560945546 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:22.936 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.931+0000 7fc1cb7fe6c0 1 --2- 10.20.193.27:0/1560945546 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc1cc115410 0x7fc1cc11d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:22.936 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.931+0000 7fc1d08066c0 1 --2- 10.20.193.27:0/1560945546 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc1cc122bb0 0x7fc1cc123050 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:22.936 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.931+0000 7fc1cbfff6c0 1 -- 10.20.193.27:0/1560945546 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc1cc115410 msgr2=0x7fc1cc11d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:22.936 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.931+0000 7fc1cbfff6c0 1 --2- 10.20.193.27:0/1560945546 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc1cc115410 0x7fc1cc11d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:22.936 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.931+0000 7fc1cbfff6c0 1 -- 10.20.193.27:0/1560945546 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc1cc122bb0 msgr2=0x7fc1cc123050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:22.936 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.931+0000 7fc1cbfff6c0 1 --2- 10.20.193.27:0/1560945546 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc1cc122bb0 0x7fc1cc123050 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:22.936 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.931+0000 7fc1cbfff6c0 1 -- 10.20.193.27:0/1560945546 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c000a660 con 0x7fc1cc10f200 2026-02-13T02:22:22.936 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.931+0000 7fc1cb7fe6c0 1 --2- 10.20.193.27:0/1560945546 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc1cc115410 0x7fc1cc11d7f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:22:22.936 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.931+0000 7fc1cbfff6c0 1 --2- 10.20.193.27:0/1560945546 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc1cc10f200 0x7fc1cc11d2b0 secure :-1 s=READY pgs=240 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc1b800b710 tx=0x7fc1b800bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:22.936 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.931+0000 7fc1c97fa6c0 1 -- 10.20.193.27:0/1560945546 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc1b8004250 con 0x7fc1cc10f200 2026-02-13T02:22:22.936 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.931+0000 7fc1d22946c0 1 -- 10.20.193.27:0/1560945546 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1cc163da0 con 0x7fc1cc10f200 2026-02-13T02:22:22.936 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.931+0000 7fc1c97fa6c0 1 -- 10.20.193.27:0/1560945546 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc1b8002d80 con 0x7fc1cc10f200 2026-02-13T02:22:22.937 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.931+0000 7fc1c97fa6c0 1 -- 10.20.193.27:0/1560945546 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc1b800ce20 con 0x7fc1cc10f200 2026-02-13T02:22:22.937 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.931+0000 7fc1d22946c0 1 -- 10.20.193.27:0/1560945546 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1cc164230 con 0x7fc1cc10f200 2026-02-13T02:22:22.937 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.932+0000 7fc1d22946c0 1 -- 10.20.193.27:0/1560945546 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1cc1104d0 con 0x7fc1cc10f200 2026-02-13T02:22:22.938 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.933+0000 7fc1c97fa6c0 1 -- 10.20.193.27:0/1560945546 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 11) ==== 157066+0+0 (secure 0 0 0) 0x7fc1b80043b0 con 0x7fc1cc10f200 2026-02-13T02:22:22.938 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.934+0000 7fc1c97fa6c0 1 --2- 10.20.193.27:0/1560945546 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc1a00b5600 0x7fc1a00d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:22.938 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.934+0000 7fc1c97fa6c0 1 -- 10.20.193.27:0/1560945546 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7fc1b80e3ab0 con 0x7fc1cc10f200 2026-02-13T02:22:22.938 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.934+0000 7fc1cb7fe6c0 1 --2- 10.20.193.27:0/1560945546 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc1a00b5600 0x7fc1a00d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:22.938 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.934+0000 7fc1c97fa6c0 1 -- 10.20.193.27:0/1560945546 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fc1b811c5e0 con 0x7fc1cc10f200 2026-02-13T02:22:22.938 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:22.934+0000 7fc1cb7fe6c0 1 --2- 10.20.193.27:0/1560945546 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc1a00b5600 0x7fc1a00d5ab0 secure :-1 s=READY pgs=175 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc1bc005e40 tx=0x7fc1bc005db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:23.046 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.045+0000 7fc1d22946c0 1 -- 10.20.193.27:0/1560945546 --> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fc1cc11e460 con 0x7fc1a00b5600 2026-02-13T02:22:23.048 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.047+0000 7fc1c97fa6c0 1 -- 10.20.193.27:0/1560945546 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7fc1b80a6250 con 0x7fc1cc10f200 2026-02-13T02:22:23.049 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.048+0000 7fc1c97fa6c0 1 -- 10.20.193.27:0/1560945546 <== mgr.14116 v2:10.20.193.27:6800/1837452956 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fc1cc11e460 con 0x7fc1a00b5600 2026-02-13T02:22:23.050 INFO:teuthology.orchestra.run.trial027.stdout:/volumes/qa/sv_1/4c9fb761-bc94-4732-a2ff-81deefc28d6f 2026-02-13T02:22:23.052 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.050+0000 7fc1d22946c0 1 -- 10.20.193.27:0/1560945546 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc1a00b5600 msgr2=0x7fc1a00d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:23.052 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.050+0000 7fc1d22946c0 1 --2- 10.20.193.27:0/1560945546 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc1a00b5600 0x7fc1a00d5ab0 secure :-1 s=READY pgs=175 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc1bc005e40 tx=0x7fc1bc005db0 comp rx=0 tx=0).stop 2026-02-13T02:22:23.052 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.050+0000 7fc1d22946c0 1 -- 10.20.193.27:0/1560945546 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc1cc10f200 msgr2=0x7fc1cc11d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:23.052 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.050+0000 7fc1d22946c0 1 --2- 10.20.193.27:0/1560945546 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc1cc10f200 0x7fc1cc11d2b0 secure :-1 s=READY pgs=240 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc1b800b710 tx=0x7fc1b800bbe0 comp rx=0 tx=0).stop 2026-02-13T02:22:23.052 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.050+0000 7fc1d22946c0 1 -- 10.20.193.27:0/1560945546 shutdown_connections 2026-02-13T02:22:23.052 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.050+0000 7fc1d22946c0 1 --2- 10.20.193.27:0/1560945546 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc1a00b5600 0x7fc1a00d5ab0 unknown :-1 s=CLOSED pgs=175 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:23.053 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.050+0000 7fc1d22946c0 1 --2- 10.20.193.27:0/1560945546 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc1cc122bb0 0x7fc1cc123050 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:23.053 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.050+0000 7fc1d22946c0 1 --2- 10.20.193.27:0/1560945546 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc1cc115410 0x7fc1cc11d7f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:23.053 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.050+0000 7fc1d22946c0 1 --2- 10.20.193.27:0/1560945546 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc1cc10f200 0x7fc1cc11d2b0 unknown :-1 s=CLOSED pgs=240 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:23.053 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.050+0000 7fc1d22946c0 1 -- 10.20.193.27:0/1560945546 >> 10.20.193.27:0/1560945546 conn(0x7fc1cc0ecb00 msgr2=0x7fc1cc10cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:23.053 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.050+0000 7fc1d22946c0 1 -- 10.20.193.27:0/1560945546 shutdown_connections 2026-02-13T02:22:23.053 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.050+0000 7fc1d22946c0 1 -- 10.20.193.27:0/1560945546 wait complete. 2026-02-13T02:22:23.091 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:23 trial027 ceph-mon[24567]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:22:23.091 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:23 trial027 ceph-mon[24567]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:22:23.092 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph fs dump --format=json 2026-02-13T02:22:23.211 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:23.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.332+0000 7f1dd1b1b6c0 1 -- 10.20.193.27:0/1385987136 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1dcc10f230 msgr2=0x7f1dcc10f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:23.336 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.332+0000 7f1dd1b1b6c0 1 --2- 10.20.193.27:0/1385987136 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1dcc10f230 0x7f1dcc10f600 secure :-1 s=READY pgs=241 gs=1 cs=0 l=1 c_cookie=7915323044728615265 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1db80099a0 tx=0x7f1db802ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:23.336 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.332+0000 7f1dd1b1b6c0 1 -- 10.20.193.27:0/1385987136 shutdown_connections 2026-02-13T02:22:23.336 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.332+0000 7f1dd1b1b6c0 1 --2- 10.20.193.27:0/1385987136 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1dcc115440 0x7f1dcc175de0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:23.336 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.332+0000 7f1dd1b1b6c0 1 --2- 10.20.193.27:0/1385987136 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1dcc10fc20 0x7f1dcc114c60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:23.336 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.332+0000 7f1dd1b1b6c0 1 --2- 10.20.193.27:0/1385987136 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1dcc10f230 0x7f1dcc10f600 unknown :-1 s=CLOSED pgs=241 gs=1 cs=0 l=1 c_cookie=7915323044728615265 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:23.336 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.332+0000 7f1dd1b1b6c0 1 -- 10.20.193.27:0/1385987136 >> 10.20.193.27:0/1385987136 conn(0x7f1dcc0ecb30 msgr2=0x7f1dcc10cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:23.336 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.332+0000 7f1dd1b1b6c0 1 -- 10.20.193.27:0/1385987136 shutdown_connections 2026-02-13T02:22:23.336 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.332+0000 7f1dd1b1b6c0 1 -- 10.20.193.27:0/1385987136 wait complete. 2026-02-13T02:22:23.337 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.332+0000 7f1dd1b1b6c0 1 Processor -- start 2026-02-13T02:22:23.337 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.332+0000 7f1dd1b1b6c0 1 -- start start 2026-02-13T02:22:23.337 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.333+0000 7f1dd1b1b6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1dcc10f230 0x7f1dcc11d270 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:23.337 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.333+0000 7f1dd1b1b6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1dcc115440 0x7f1dcc11d7b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:23.337 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.333+0000 7f1dd1b1b6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1dcc122b70 0x7f1dcc123010 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:23.337 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.333+0000 7f1dd1b1b6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1dcc123580 con 0x7f1dcc10f230 2026-02-13T02:22:23.337 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.333+0000 7f1dd1b1b6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1dcc1236f0 con 0x7f1dcc122b70 2026-02-13T02:22:23.337 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.333+0000 7f1dd1b1b6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1dcc1238b0 con 0x7f1dcc115440 2026-02-13T02:22:23.337 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.333+0000 7f1dcb7fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1dcc10f230 0x7f1dcc11d270 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:23.337 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.333+0000 7f1dcb7fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1dcc10f230 0x7f1dcc11d270 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55364/0 (socket says 10.20.193.27:55364) 2026-02-13T02:22:23.337 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.333+0000 7f1dcb7fe6c0 1 -- 10.20.193.27:0/4030777049 learned_addr learned my addr 10.20.193.27:0/4030777049 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:23.338 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.333+0000 7f1dcaffd6c0 1 --2- 10.20.193.27:0/4030777049 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1dcc115440 0x7f1dcc11d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:23.338 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.333+0000 7f1dcb7fe6c0 1 -- 10.20.193.27:0/4030777049 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1dcc115440 msgr2=0x7f1dcc11d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:23.338 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.333+0000 7f1dcb7fe6c0 1 --2- 10.20.193.27:0/4030777049 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1dcc115440 0x7f1dcc11d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:23.338 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.333+0000 7f1dcbfff6c0 1 --2- 10.20.193.27:0/4030777049 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1dcc122b70 0x7f1dcc123010 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:23.338 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.333+0000 7f1dcb7fe6c0 1 -- 10.20.193.27:0/4030777049 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1dcc122b70 msgr2=0x7f1dcc123010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:23.338 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.333+0000 7f1dcb7fe6c0 1 --2- 10.20.193.27:0/4030777049 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1dcc122b70 0x7f1dcc123010 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:23.338 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.333+0000 7f1dcb7fe6c0 1 -- 10.20.193.27:0/4030777049 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db8009590 con 0x7f1dcc10f230 2026-02-13T02:22:23.338 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.333+0000 7f1dcbfff6c0 1 --2- 10.20.193.27:0/4030777049 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1dcc122b70 0x7f1dcc123010 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:22:23.338 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.333+0000 7f1dcb7fe6c0 1 --2- 10.20.193.27:0/4030777049 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1dcc10f230 0x7f1dcc11d270 secure :-1 s=READY pgs=242 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1db802f230 tx=0x7f1db802fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:23.338 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.333+0000 7f1dc8ff96c0 1 -- 10.20.193.27:0/4030777049 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1db8002ae0 con 0x7f1dcc10f230 2026-02-13T02:22:23.338 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.333+0000 7f1dc8ff96c0 1 -- 10.20.193.27:0/4030777049 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1db8007680 con 0x7f1dcc10f230 2026-02-13T02:22:23.338 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.333+0000 7f1dd1b1b6c0 1 -- 10.20.193.27:0/4030777049 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dcc163d00 con 0x7f1dcc10f230 2026-02-13T02:22:23.339 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.333+0000 7f1dc8ff96c0 1 -- 10.20.193.27:0/4030777049 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1db8037520 con 0x7f1dcc10f230 2026-02-13T02:22:23.339 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.333+0000 7f1dd1b1b6c0 1 -- 10.20.193.27:0/4030777049 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dcc164130 con 0x7f1dcc10f230 2026-02-13T02:22:23.339 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.333+0000 7f1dd1b1b6c0 1 -- 10.20.193.27:0/4030777049 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dcc110500 con 0x7f1dcc10f230 2026-02-13T02:22:23.340 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.335+0000 7f1dc8ff96c0 1 -- 10.20.193.27:0/4030777049 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f1db8007cb0 con 0x7f1dcc10f230 2026-02-13T02:22:23.340 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.335+0000 7f1dc8ff96c0 1 --2- 10.20.193.27:0/4030777049 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f1da00b5690 0x7f1da00d5b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:23.340 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.335+0000 7f1dc8ff96c0 1 -- 10.20.193.27:0/4030777049 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f1db8107da0 con 0x7f1dcc10f230 2026-02-13T02:22:23.340 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.335+0000 7f1dcaffd6c0 1 --2- 10.20.193.27:0/4030777049 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f1da00b5690 0x7f1da00d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:23.340 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.336+0000 7f1dc8ff96c0 1 -- 10.20.193.27:0/4030777049 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f1db80ca800 con 0x7f1dcc10f230 2026-02-13T02:22:23.340 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.336+0000 7f1dcaffd6c0 1 --2- 10.20.193.27:0/4030777049 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f1da00b5690 0x7f1da00d5b40 secure :-1 s=READY pgs=176 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1dbc005980 tx=0x7f1dbc005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:23.451 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.450+0000 7f1dd1b1b6c0 1 -- 10.20.193.27:0/4030777049 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1dcc10fc20 con 0x7f1dcc10f230 2026-02-13T02:22:23.452 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.451+0000 7f1dc8ff96c0 1 -- 10.20.193.27:0/4030777049 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11872 (secure 0 0 0) 0x7f1db80cd9b0 con 0x7f1dcc10f230 2026-02-13T02:22:23.453 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:22:23.453 INFO:teuthology.orchestra.run.trial027.stdout:{"epoch":18,"btime":"2026-02-13T02:22:21:032561+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.27:6835/3709761799","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6834","nonce":3709761799},{"type":"v1","addr":"10.20.193.27:6835","nonce":3709761799}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.27:6837/747672540","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6836","nonce":747672540},{"type":"v1","addr":"10.20.193.27:6837","nonce":747672540}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14730,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.27:6839/761293857","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6838","nonce":761293857},{"type":"v1","addr":"10.20.193.27:6839","nonce":761293857}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":14736,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.27:6841/1659204922","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6840","nonce":1659204922},{"type":"v1","addr":"10.20.193.27:6841","nonce":1659204922}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.107:6833/3112510413","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6832","nonce":3112510413},{"type":"v1","addr":"10.20.193.107:6833","nonce":3112510413}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6833/2774006818","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6832","nonce":2774006818},{"type":"v1","addr":"10.20.193.28:6833","nonce":2774006818}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.107:6835/3281057176","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6834","nonce":3281057176},{"type":"v1","addr":"10.20.193.107:6835","nonce":3281057176}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.107:6837/2849542469","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6836","nonce":2849542469},{"type":"v1","addr":"10.20.193.107:6837","nonce":2849542469}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/1510901141","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":1510901141},{"type":"v1","addr":"10.20.193.28:6835","nonce":1510901141}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.107:6839/3260223906","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6838","nonce":3260223906},{"type":"v1","addr":"10.20.193.107:6839","nonce":3260223906}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-13T02:22:16.405975+0000","modified":"2026-02-13T02:22:20.416749+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.28:6837/543345874","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":543345874},{"type":"v1","addr":"10.20.193.28:6837","nonce":543345874}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"10.20.193.28:6839/2398862063","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":2398862063},{"type":"v1","addr":"10.20.193.28:6839","nonce":2398862063}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491]},"id":1}]} 2026-02-13T02:22:23.454 INFO:teuthology.orchestra.run.trial027.stderr:dumped fsmap epoch 18 2026-02-13T02:22:23.454 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.451+0000 7f1dd1b1b6c0 1 -- 10.20.193.27:0/4030777049 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f1da00b5690 msgr2=0x7f1da00d5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:23.454 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.451+0000 7f1dd1b1b6c0 1 --2- 10.20.193.27:0/4030777049 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f1da00b5690 0x7f1da00d5b40 secure :-1 s=READY pgs=176 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1dbc005980 tx=0x7f1dbc005910 comp rx=0 tx=0).stop 2026-02-13T02:22:23.454 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.451+0000 7f1dd1b1b6c0 1 -- 10.20.193.27:0/4030777049 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1dcc10f230 msgr2=0x7f1dcc11d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:23.454 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.451+0000 7f1dd1b1b6c0 1 --2- 10.20.193.27:0/4030777049 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1dcc10f230 0x7f1dcc11d270 secure :-1 s=READY pgs=242 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1db802f230 tx=0x7f1db802fc00 comp rx=0 tx=0).stop 2026-02-13T02:22:23.454 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.452+0000 7f1dd1b1b6c0 1 -- 10.20.193.27:0/4030777049 shutdown_connections 2026-02-13T02:22:23.455 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.452+0000 7f1dd1b1b6c0 1 --2- 10.20.193.27:0/4030777049 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f1da00b5690 0x7f1da00d5b40 unknown :-1 s=CLOSED pgs=176 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:23.455 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.452+0000 7f1dd1b1b6c0 1 --2- 10.20.193.27:0/4030777049 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1dcc122b70 0x7f1dcc123010 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:23.455 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.452+0000 7f1dd1b1b6c0 1 --2- 10.20.193.27:0/4030777049 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1dcc115440 0x7f1dcc11d7b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:23.455 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.452+0000 7f1dd1b1b6c0 1 --2- 10.20.193.27:0/4030777049 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1dcc10f230 0x7f1dcc11d270 unknown :-1 s=CLOSED pgs=242 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:23.455 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.452+0000 7f1dd1b1b6c0 1 -- 10.20.193.27:0/4030777049 >> 10.20.193.27:0/4030777049 conn(0x7f1dcc0ecb30 msgr2=0x7f1dcc10cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:23.455 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.452+0000 7f1dd1b1b6c0 1 -- 10.20.193.27:0/4030777049 shutdown_connections 2026-02-13T02:22:23.455 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.452+0000 7f1dd1b1b6c0 1 -- 10.20.193.27:0/4030777049 wait complete. 2026-02-13T02:22:23.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:23 trial028 ceph-mon[26701]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:22:23.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:23 trial028 ceph-mon[26701]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:22:23.495 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd dump --format=json 2026-02-13T02:22:23.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:23 trial107 ceph-mon[27422]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:22:23.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:23 trial107 ceph-mon[27422]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:22:23.611 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:23.731 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.728+0000 7f599c3ff6c0 1 -- 10.20.193.27:0/1148956973 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f599410fbf0 msgr2=0x7f5994114c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:23.731 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.728+0000 7f599c3ff6c0 1 --2- 10.20.193.27:0/1148956973 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f599410fbf0 0x7f5994114c30 secure :-1 s=READY pgs=243 gs=1 cs=0 l=1 c_cookie=2464340505306114869 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f59880099a0 tx=0x7f598802ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:23.731 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.728+0000 7f599c3ff6c0 1 -- 10.20.193.27:0/1148956973 shutdown_connections 2026-02-13T02:22:23.732 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.728+0000 7f599c3ff6c0 1 --2- 10.20.193.27:0/1148956973 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5994115410 0x7f5994175db0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:23.732 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.728+0000 7f599c3ff6c0 1 --2- 10.20.193.27:0/1148956973 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f599410fbf0 0x7f5994114c30 unknown :-1 s=CLOSED pgs=243 gs=1 cs=0 l=1 c_cookie=2464340505306114869 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:23.732 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.728+0000 7f599c3ff6c0 1 --2- 10.20.193.27:0/1148956973 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f599410f200 0x7f599410f5d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:23.732 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.728+0000 7f599c3ff6c0 1 -- 10.20.193.27:0/1148956973 >> 10.20.193.27:0/1148956973 conn(0x7f59940ecb00 msgr2=0x7f599410cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:23.732 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.728+0000 7f599c3ff6c0 1 -- 10.20.193.27:0/1148956973 shutdown_connections 2026-02-13T02:22:23.732 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.728+0000 7f599c3ff6c0 1 -- 10.20.193.27:0/1148956973 wait complete. 2026-02-13T02:22:23.732 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.728+0000 7f599c3ff6c0 1 Processor -- start 2026-02-13T02:22:23.732 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.728+0000 7f599c3ff6c0 1 -- start start 2026-02-13T02:22:23.732 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.728+0000 7f599c3ff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f599410f200 0x7f59940f4d00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:23.732 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.728+0000 7f599c3ff6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5994115410 0x7f5994175cb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:23.733 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.728+0000 7f599c3ff6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f59940f5330 0x7f59940fa5b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:23.733 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.728+0000 7f599c3ff6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f59940f5a20 con 0x7f599410f200 2026-02-13T02:22:23.733 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.728+0000 7f599c3ff6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f59940faaf0 con 0x7f59940f5330 2026-02-13T02:22:23.733 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.728+0000 7f599c3ff6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f59940facb0 con 0x7f5994115410 2026-02-13T02:22:23.733 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.728+0000 7f599a1706c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f599410f200 0x7f59940f4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:23.733 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.728+0000 7f599a1706c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f599410f200 0x7f59940f4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55382/0 (socket says 10.20.193.27:55382) 2026-02-13T02:22:23.733 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.728+0000 7f599a1706c0 1 -- 10.20.193.27:0/4281799962 learned_addr learned my addr 10.20.193.27:0/4281799962 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:23.733 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.728+0000 7f599996f6c0 1 --2- 10.20.193.27:0/4281799962 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5994115410 0x7f5994175cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:23.733 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.728+0000 7f599a1706c0 1 -- 10.20.193.27:0/4281799962 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5994115410 msgr2=0x7f5994175cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:23.733 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.728+0000 7f599a9716c0 1 --2- 10.20.193.27:0/4281799962 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f59940f5330 0x7f59940fa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:23.733 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.728+0000 7f599a1706c0 1 --2- 10.20.193.27:0/4281799962 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5994115410 0x7f5994175cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:23.733 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.728+0000 7f599a1706c0 1 -- 10.20.193.27:0/4281799962 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f59940f5330 msgr2=0x7f59940fa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:23.734 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.728+0000 7f599a1706c0 1 --2- 10.20.193.27:0/4281799962 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f59940f5330 0x7f59940fa5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:23.734 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.728+0000 7f599a1706c0 1 -- 10.20.193.27:0/4281799962 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5988009590 con 0x7f599410f200 2026-02-13T02:22:23.734 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.728+0000 7f599996f6c0 1 --2- 10.20.193.27:0/4281799962 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5994115410 0x7f5994175cb0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:22:23.734 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.728+0000 7f599a9716c0 1 --2- 10.20.193.27:0/4281799962 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f59940f5330 0x7f59940fa5b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:22:23.734 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.729+0000 7f599a1706c0 1 --2- 10.20.193.27:0/4281799962 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f599410f200 0x7f59940f4d00 secure :-1 s=READY pgs=244 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f598400b710 tx=0x7f598400bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:23.734 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.729+0000 7f597b7fe6c0 1 -- 10.20.193.27:0/4281799962 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5984004250 con 0x7f599410f200 2026-02-13T02:22:23.734 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.729+0000 7f597b7fe6c0 1 -- 10.20.193.27:0/4281799962 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5984009070 con 0x7f599410f200 2026-02-13T02:22:23.734 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.729+0000 7f599c3ff6c0 1 -- 10.20.193.27:0/4281799962 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59940faf90 con 0x7f599410f200 2026-02-13T02:22:23.734 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.729+0000 7f597b7fe6c0 1 -- 10.20.193.27:0/4281799962 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f598400ccd0 con 0x7f599410f200 2026-02-13T02:22:23.734 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.729+0000 7f599c3ff6c0 1 -- 10.20.193.27:0/4281799962 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59940fb720 con 0x7f599410f200 2026-02-13T02:22:23.734 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.729+0000 7f599c3ff6c0 1 -- 10.20.193.27:0/4281799962 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59941104d0 con 0x7f599410f200 2026-02-13T02:22:23.736 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.730+0000 7f597b7fe6c0 1 -- 10.20.193.27:0/4281799962 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f59840028c0 con 0x7f599410f200 2026-02-13T02:22:23.736 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.731+0000 7f597b7fe6c0 1 --2- 10.20.193.27:0/4281799962 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f59700b5710 0x7f59700d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:23.736 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.731+0000 7f597b7fe6c0 1 -- 10.20.193.27:0/4281799962 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f59840e3be0 con 0x7f599410f200 2026-02-13T02:22:23.736 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.731+0000 7f599996f6c0 1 --2- 10.20.193.27:0/4281799962 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f59700b5710 0x7f59700d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:23.736 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.731+0000 7f599996f6c0 1 --2- 10.20.193.27:0/4281799962 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f59700b5710 0x7f59700d5bc0 secure :-1 s=READY pgs=177 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f59880098c0 tx=0x7f598802f580 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:23.736 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.732+0000 7f597b7fe6c0 1 -- 10.20.193.27:0/4281799962 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f59840a6640 con 0x7f599410f200 2026-02-13T02:22:23.824 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.823+0000 7f599c3ff6c0 1 -- 10.20.193.27:0/4281799962 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59940f5e60 con 0x7f599410f200 2026-02-13T02:22:23.825 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.823+0000 7f597b7fe6c0 1 -- 10.20.193.27:0/4281799962 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+22002 (secure 0 0 0) 0x7f598400ce60 con 0x7f599410f200 2026-02-13T02:22:23.825 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:22:23.826 INFO:teuthology.orchestra.run.trial027.stdout:{"epoch":70,"fsid":"707dbb29-0882-11f1-9262-d404e6e7d460","created":"2026-02-13T02:19:50.420575+0000","modified":"2026-02-13T02:22:18.009553+0000","last_up_change":"2026-02-13T02:21:58.960121+0000","last_in_change":"2026-02-13T02:21:50.695981+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-13T02:20:54.702258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2026-02-13T02:22:14.280021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2026-02-13T02:22:15.383271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1a7af29f-2999-4be3-a38e-3fc7feb9f2f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":7,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6802","nonce":832083434},{"type":"v1","addr":"10.20.193.27:6803","nonce":832083434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6804","nonce":832083434},{"type":"v1","addr":"10.20.193.27:6805","nonce":832083434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6808","nonce":832083434},{"type":"v1","addr":"10.20.193.27:6809","nonce":832083434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6806","nonce":832083434},{"type":"v1","addr":"10.20.193.27:6807","nonce":832083434}]},"public_addr":"10.20.193.27:6803/832083434","cluster_addr":"10.20.193.27:6805/832083434","heartbeat_back_addr":"10.20.193.27:6809/832083434","heartbeat_front_addr":"10.20.193.27:6807/832083434","state":["exists","up"]},{"osd":1,"uuid":"8c0ad8cf-4150-43f6-8a9c-258b0ebf6a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6800","nonce":4070516162},{"type":"v1","addr":"10.20.193.28:6801","nonce":4070516162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6802","nonce":4070516162},{"type":"v1","addr":"10.20.193.28:6803","nonce":4070516162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6806","nonce":4070516162},{"type":"v1","addr":"10.20.193.28:6807","nonce":4070516162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6804","nonce":4070516162},{"type":"v1","addr":"10.20.193.28:6805","nonce":4070516162}]},"public_addr":"10.20.193.28:6801/4070516162","cluster_addr":"10.20.193.28:6803/4070516162","heartbeat_back_addr":"10.20.193.28:6807/4070516162","heartbeat_front_addr":"10.20.193.28:6805/4070516162","state":["exists","up"]},{"osd":2,"uuid":"a7813078-94d6-4559-a89a-5525161ab5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6800","nonce":4075625103},{"type":"v1","addr":"10.20.193.107:6801","nonce":4075625103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6802","nonce":4075625103},{"type":"v1","addr":"10.20.193.107:6803","nonce":4075625103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6806","nonce":4075625103},{"type":"v1","addr":"10.20.193.107:6807","nonce":4075625103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6804","nonce":4075625103},{"type":"v1","addr":"10.20.193.107:6805","nonce":4075625103}]},"public_addr":"10.20.193.107:6801/4075625103","cluster_addr":"10.20.193.107:6803/4075625103","heartbeat_back_addr":"10.20.193.107:6807/4075625103","heartbeat_front_addr":"10.20.193.107:6805/4075625103","state":["exists","up"]},{"osd":3,"uuid":"268f9714-faed-452b-ae34-3240fa23153a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6810","nonce":382071678},{"type":"v1","addr":"10.20.193.27:6811","nonce":382071678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6812","nonce":382071678},{"type":"v1","addr":"10.20.193.27:6813","nonce":382071678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6816","nonce":382071678},{"type":"v1","addr":"10.20.193.27:6817","nonce":382071678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6814","nonce":382071678},{"type":"v1","addr":"10.20.193.27:6815","nonce":382071678}]},"public_addr":"10.20.193.27:6811/382071678","cluster_addr":"10.20.193.27:6813/382071678","heartbeat_back_addr":"10.20.193.27:6817/382071678","heartbeat_front_addr":"10.20.193.27:6815/382071678","state":["exists","up"]},{"osd":4,"uuid":"4beaec0d-7b39-4bb0-9770-3dc4c848b106","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6808","nonce":711804032},{"type":"v1","addr":"10.20.193.28:6809","nonce":711804032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6810","nonce":711804032},{"type":"v1","addr":"10.20.193.28:6811","nonce":711804032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6814","nonce":711804032},{"type":"v1","addr":"10.20.193.28:6815","nonce":711804032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6812","nonce":711804032},{"type":"v1","addr":"10.20.193.28:6813","nonce":711804032}]},"public_addr":"10.20.193.28:6809/711804032","cluster_addr":"10.20.193.28:6811/711804032","heartbeat_back_addr":"10.20.193.28:6815/711804032","heartbeat_front_addr":"10.20.193.28:6813/711804032","state":["exists","up"]},{"osd":5,"uuid":"bb16754e-66f2-4951-9f31-d1dde591ec92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6808","nonce":4246100128},{"type":"v1","addr":"10.20.193.107:6809","nonce":4246100128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6810","nonce":4246100128},{"type":"v1","addr":"10.20.193.107:6811","nonce":4246100128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6814","nonce":4246100128},{"type":"v1","addr":"10.20.193.107:6815","nonce":4246100128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6812","nonce":4246100128},{"type":"v1","addr":"10.20.193.107:6813","nonce":4246100128}]},"public_addr":"10.20.193.107:6809/4246100128","cluster_addr":"10.20.193.107:6811/4246100128","heartbeat_back_addr":"10.20.193.107:6815/4246100128","heartbeat_front_addr":"10.20.193.107:6813/4246100128","state":["exists","up"]},{"osd":6,"uuid":"318afff7-b7c9-4b19-a7ba-06e7ab122ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6818","nonce":3514139178},{"type":"v1","addr":"10.20.193.27:6819","nonce":3514139178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6820","nonce":3514139178},{"type":"v1","addr":"10.20.193.27:6821","nonce":3514139178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6824","nonce":3514139178},{"type":"v1","addr":"10.20.193.27:6825","nonce":3514139178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6822","nonce":3514139178},{"type":"v1","addr":"10.20.193.27:6823","nonce":3514139178}]},"public_addr":"10.20.193.27:6819/3514139178","cluster_addr":"10.20.193.27:6821/3514139178","heartbeat_back_addr":"10.20.193.27:6825/3514139178","heartbeat_front_addr":"10.20.193.27:6823/3514139178","state":["exists","up"]},{"osd":7,"uuid":"b545439a-3f32-4b0a-bc6a-17f0d8050c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6816","nonce":1405200379},{"type":"v1","addr":"10.20.193.28:6817","nonce":1405200379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6818","nonce":1405200379},{"type":"v1","addr":"10.20.193.28:6819","nonce":1405200379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6822","nonce":1405200379},{"type":"v1","addr":"10.20.193.28:6823","nonce":1405200379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6820","nonce":1405200379},{"type":"v1","addr":"10.20.193.28:6821","nonce":1405200379}]},"public_addr":"10.20.193.28:6817/1405200379","cluster_addr":"10.20.193.28:6819/1405200379","heartbeat_back_addr":"10.20.193.28:6823/1405200379","heartbeat_front_addr":"10.20.193.28:6821/1405200379","state":["exists","up"]},{"osd":8,"uuid":"a7c6bbba-760c-484a-8171-44c70f7d9c89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6816","nonce":1331562104},{"type":"v1","addr":"10.20.193.107:6817","nonce":1331562104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6818","nonce":1331562104},{"type":"v1","addr":"10.20.193.107:6819","nonce":1331562104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6822","nonce":1331562104},{"type":"v1","addr":"10.20.193.107:6823","nonce":1331562104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6820","nonce":1331562104},{"type":"v1","addr":"10.20.193.107:6821","nonce":1331562104}]},"public_addr":"10.20.193.107:6817/1331562104","cluster_addr":"10.20.193.107:6819/1331562104","heartbeat_back_addr":"10.20.193.107:6823/1331562104","heartbeat_front_addr":"10.20.193.107:6821/1331562104","state":["exists","up"]},{"osd":9,"uuid":"72cbd422-3c3a-4904-9075-f543671d4f48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6826","nonce":1444927853},{"type":"v1","addr":"10.20.193.27:6827","nonce":1444927853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6828","nonce":1444927853},{"type":"v1","addr":"10.20.193.27:6829","nonce":1444927853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6832","nonce":1444927853},{"type":"v1","addr":"10.20.193.27:6833","nonce":1444927853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6830","nonce":1444927853},{"type":"v1","addr":"10.20.193.27:6831","nonce":1444927853}]},"public_addr":"10.20.193.27:6827/1444927853","cluster_addr":"10.20.193.27:6829/1444927853","heartbeat_back_addr":"10.20.193.27:6833/1444927853","heartbeat_front_addr":"10.20.193.27:6831/1444927853","state":["exists","up"]},{"osd":10,"uuid":"4bb7b75e-adf7-4c38-9343-dba513b57ed7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6824","nonce":2255755797},{"type":"v1","addr":"10.20.193.28:6825","nonce":2255755797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6826","nonce":2255755797},{"type":"v1","addr":"10.20.193.28:6827","nonce":2255755797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6830","nonce":2255755797},{"type":"v1","addr":"10.20.193.28:6831","nonce":2255755797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6828","nonce":2255755797},{"type":"v1","addr":"10.20.193.28:6829","nonce":2255755797}]},"public_addr":"10.20.193.28:6825/2255755797","cluster_addr":"10.20.193.28:6827/2255755797","heartbeat_back_addr":"10.20.193.28:6831/2255755797","heartbeat_front_addr":"10.20.193.28:6829/2255755797","state":["exists","up"]},{"osd":11,"uuid":"115fe8c6-77fc-41dd-ade0-db8eea024974","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6824","nonce":1496433214},{"type":"v1","addr":"10.20.193.107:6825","nonce":1496433214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6826","nonce":1496433214},{"type":"v1","addr":"10.20.193.107:6827","nonce":1496433214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6830","nonce":1496433214},{"type":"v1","addr":"10.20.193.107:6831","nonce":1496433214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6828","nonce":1496433214},{"type":"v1","addr":"10.20.193.107:6829","nonce":1496433214}]},"public_addr":"10.20.193.107:6825/1496433214","cluster_addr":"10.20.193.107:6827/1496433214","heartbeat_back_addr":"10.20.193.107:6831/1496433214","heartbeat_front_addr":"10.20.193.107:6829/1496433214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:20:37.596432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:20:44.867765+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:20:51.765273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:20:58.994342+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:06.232104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:13.195531+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:20.453172+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:27.721147+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:35.051979+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:42.463998+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:49.889642+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:57.167778+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"10.20.193.27:0/1486586326":"2026-02-14T02:19:56.309429+0000","10.20.193.27:6801/3576730225":"2026-02-14T02:19:56.309429+0000","10.20.193.27:6800/3576730225":"2026-02-14T02:19:56.309429+0000","10.20.193.27:0/2609227888":"2026-02-14T02:19:56.309429+0000","10.20.193.27:0/1389740324":"2026-02-14T02:19:56.309429+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-13T02:22:23.827 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.824+0000 7f599c3ff6c0 1 -- 10.20.193.27:0/4281799962 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f59700b5710 msgr2=0x7f59700d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:23.827 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.824+0000 7f599c3ff6c0 1 --2- 10.20.193.27:0/4281799962 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f59700b5710 0x7f59700d5bc0 secure :-1 s=READY pgs=177 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f59880098c0 tx=0x7f598802f580 comp rx=0 tx=0).stop 2026-02-13T02:22:23.827 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.824+0000 7f599c3ff6c0 1 -- 10.20.193.27:0/4281799962 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f599410f200 msgr2=0x7f59940f4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:23.827 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.824+0000 7f599c3ff6c0 1 --2- 10.20.193.27:0/4281799962 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f599410f200 0x7f59940f4d00 secure :-1 s=READY pgs=244 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f598400b710 tx=0x7f598400bbe0 comp rx=0 tx=0).stop 2026-02-13T02:22:23.827 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.824+0000 7f599c3ff6c0 1 -- 10.20.193.27:0/4281799962 shutdown_connections 2026-02-13T02:22:23.827 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.824+0000 7f599c3ff6c0 1 --2- 10.20.193.27:0/4281799962 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f59700b5710 0x7f59700d5bc0 unknown :-1 s=CLOSED pgs=177 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:23.827 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.824+0000 7f599c3ff6c0 1 --2- 10.20.193.27:0/4281799962 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f59940f5330 0x7f59940fa5b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:23.827 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.824+0000 7f599c3ff6c0 1 --2- 10.20.193.27:0/4281799962 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5994115410 0x7f5994175cb0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:23.827 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.824+0000 7f599c3ff6c0 1 --2- 10.20.193.27:0/4281799962 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f599410f200 0x7f59940f4d00 unknown :-1 s=CLOSED pgs=244 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:23.827 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.824+0000 7f599c3ff6c0 1 -- 10.20.193.27:0/4281799962 >> 10.20.193.27:0/4281799962 conn(0x7f59940ecb00 msgr2=0x7f599410d940 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:23.828 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.824+0000 7f599c3ff6c0 1 -- 10.20.193.27:0/4281799962 shutdown_connections 2026-02-13T02:22:23.828 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:23.824+0000 7f599c3ff6c0 1 -- 10.20.193.27:0/4281799962 wait complete. 2026-02-13T02:22:23.867 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2026-02-13T02:22:23.868 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph pg dump --format=json 2026-02-13T02:22:23.984 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:24.103 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.100+0000 7f11c21b56c0 1 -- 10.20.193.27:0/3789797584 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f11bc060720 msgr2=0x7f11bc060af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:24.103 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.100+0000 7f11c21b56c0 1 --2- 10.20.193.27:0/3789797584 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f11bc060720 0x7f11bc060af0 secure :-1 s=READY pgs=245 gs=1 cs=0 l=1 c_cookie=8889688073466190290 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f11a8009970 tx=0x7f11a802ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:24.103 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.100+0000 7f11c21b56c0 1 -- 10.20.193.27:0/3789797584 shutdown_connections 2026-02-13T02:22:24.104 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.100+0000 7f11c21b56c0 1 --2- 10.20.193.27:0/3789797584 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f11bc1556e0 0x7f11bc175ac0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:24.104 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.100+0000 7f11c21b56c0 1 --2- 10.20.193.27:0/3789797584 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f11bc061030 0x7f11bc1551a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:24.104 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.100+0000 7f11c21b56c0 1 --2- 10.20.193.27:0/3789797584 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f11bc060720 0x7f11bc060af0 unknown :-1 s=CLOSED pgs=245 gs=1 cs=0 l=1 c_cookie=8889688073466190290 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:24.104 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.100+0000 7f11c21b56c0 1 -- 10.20.193.27:0/3789797584 >> 10.20.193.27:0/3789797584 conn(0x7f11bc0ea8b0 msgr2=0x7f11bc10acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:24.104 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.100+0000 7f11c21b56c0 1 -- 10.20.193.27:0/3789797584 shutdown_connections 2026-02-13T02:22:24.104 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.100+0000 7f11c21b56c0 1 -- 10.20.193.27:0/3789797584 wait complete. 2026-02-13T02:22:24.104 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.100+0000 7f11c21b56c0 1 Processor -- start 2026-02-13T02:22:24.104 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.100+0000 7f11c21b56c0 1 -- start start 2026-02-13T02:22:24.104 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.100+0000 7f11c21b56c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f11bc060720 0x7f11bc1615b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:24.104 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.100+0000 7f11c21b56c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f11bc061030 0x7f11bc161af0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:24.105 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.100+0000 7f11c21b56c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f11bc1556e0 0x7f11bc166eb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:24.105 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.100+0000 7f11c21b56c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc167500 con 0x7f11bc060720 2026-02-13T02:22:24.105 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.100+0000 7f11c21b56c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc167670 con 0x7f11bc061030 2026-02-13T02:22:24.105 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.100+0000 7f11c21b56c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc167830 con 0x7f11bc1556e0 2026-02-13T02:22:24.105 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.100+0000 7f11bb7fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f11bc060720 0x7f11bc1615b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:24.105 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.100+0000 7f11bb7fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f11bc060720 0x7f11bc1615b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55404/0 (socket says 10.20.193.27:55404) 2026-02-13T02:22:24.105 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.100+0000 7f11bb7fe6c0 1 -- 10.20.193.27:0/478977441 learned_addr learned my addr 10.20.193.27:0/478977441 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:24.105 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.100+0000 7f11baffd6c0 1 --2- 10.20.193.27:0/478977441 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f11bc061030 0x7f11bc161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:24.105 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.100+0000 7f11bbfff6c0 1 --2- 10.20.193.27:0/478977441 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f11bc1556e0 0x7f11bc166eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:24.105 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.100+0000 7f11bb7fe6c0 1 -- 10.20.193.27:0/478977441 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f11bc1556e0 msgr2=0x7f11bc166eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:24.105 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.100+0000 7f11bb7fe6c0 1 --2- 10.20.193.27:0/478977441 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f11bc1556e0 0x7f11bc166eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:24.106 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.100+0000 7f11bb7fe6c0 1 -- 10.20.193.27:0/478977441 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f11bc061030 msgr2=0x7f11bc161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:24.106 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.101+0000 7f11bb7fe6c0 1 --2- 10.20.193.27:0/478977441 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f11bc061030 0x7f11bc161af0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:24.106 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.101+0000 7f11bb7fe6c0 1 -- 10.20.193.27:0/478977441 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a8009590 con 0x7f11bc060720 2026-02-13T02:22:24.106 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.101+0000 7f11bb7fe6c0 1 --2- 10.20.193.27:0/478977441 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f11bc060720 0x7f11bc1615b0 secure :-1 s=READY pgs=246 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f11a80092b0 tx=0x7f11a802fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:24.106 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.101+0000 7f11b8ff96c0 1 -- 10.20.193.27:0/478977441 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f11a8002ae0 con 0x7f11bc060720 2026-02-13T02:22:24.106 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.101+0000 7f11b8ff96c0 1 -- 10.20.193.27:0/478977441 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f11a8007680 con 0x7f11bc060720 2026-02-13T02:22:24.106 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.101+0000 7f11c21b56c0 1 -- 10.20.193.27:0/478977441 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11bc167ab0 con 0x7f11bc060720 2026-02-13T02:22:24.106 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.101+0000 7f11b8ff96c0 1 -- 10.20.193.27:0/478977441 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f11a8037520 con 0x7f11bc060720 2026-02-13T02:22:24.106 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.101+0000 7f11c21b56c0 1 -- 10.20.193.27:0/478977441 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11bc151310 con 0x7f11bc060720 2026-02-13T02:22:24.106 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.101+0000 7f11c21b56c0 1 -- 10.20.193.27:0/478977441 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11bc061c30 con 0x7f11bc060720 2026-02-13T02:22:24.107 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.103+0000 7f11b8ff96c0 1 -- 10.20.193.27:0/478977441 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f11a8007cb0 con 0x7f11bc060720 2026-02-13T02:22:24.108 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.103+0000 7f11b8ff96c0 1 --2- 10.20.193.27:0/478977441 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f11900b5690 0x7f11900d5b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:24.108 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.103+0000 7f11baffd6c0 1 --2- 10.20.193.27:0/478977441 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f11900b5690 0x7f11900d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:24.108 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.104+0000 7f11b8ff96c0 1 -- 10.20.193.27:0/478977441 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f11a8107cb0 con 0x7f11bc060720 2026-02-13T02:22:24.108 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.104+0000 7f11baffd6c0 1 --2- 10.20.193.27:0/478977441 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f11900b5690 0x7f11900d5b40 secure :-1 s=READY pgs=178 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f11bc167f40 tx=0x7f11ac009390 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:24.108 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.104+0000 7f11b8ff96c0 1 -- 10.20.193.27:0/478977441 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f11a80cd880 con 0x7f11bc060720 2026-02-13T02:22:24.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.192+0000 7f11c21b56c0 1 -- 10.20.193.27:0/478977441 --> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f11bc060b80 con 0x7f11900b5690 2026-02-13T02:22:24.195 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.194+0000 7f11b8ff96c0 1 -- 10.20.193.27:0/478977441 <== mgr.14116 v2:10.20.193.27:6800/1837452956 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258825 (secure 0 0 0) 0x7f11ac00bae0 con 0x7f11900b5690 2026-02-13T02:22:24.198 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:22:24.200 INFO:teuthology.orchestra.run.trial027.stderr:dumped all 2026-02-13T02:22:24.202 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.195+0000 7f11c21b56c0 1 -- 10.20.193.27:0/478977441 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f11900b5690 msgr2=0x7f11900d5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:24.203 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.195+0000 7f11c21b56c0 1 --2- 10.20.193.27:0/478977441 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f11900b5690 0x7f11900d5b40 secure :-1 s=READY pgs=178 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f11bc167f40 tx=0x7f11ac009390 comp rx=0 tx=0).stop 2026-02-13T02:22:24.203 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.195+0000 7f11c21b56c0 1 -- 10.20.193.27:0/478977441 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f11bc060720 msgr2=0x7f11bc1615b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:24.203 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.195+0000 7f11c21b56c0 1 --2- 10.20.193.27:0/478977441 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f11bc060720 0x7f11bc1615b0 secure :-1 s=READY pgs=246 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f11a80092b0 tx=0x7f11a802fc00 comp rx=0 tx=0).stop 2026-02-13T02:22:24.203 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.195+0000 7f11c21b56c0 1 -- 10.20.193.27:0/478977441 shutdown_connections 2026-02-13T02:22:24.203 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.195+0000 7f11c21b56c0 1 --2- 10.20.193.27:0/478977441 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f11900b5690 0x7f11900d5b40 unknown :-1 s=CLOSED pgs=178 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:24.203 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.195+0000 7f11c21b56c0 1 --2- 10.20.193.27:0/478977441 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f11bc1556e0 0x7f11bc166eb0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:24.203 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.195+0000 7f11c21b56c0 1 --2- 10.20.193.27:0/478977441 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f11bc061030 0x7f11bc161af0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:24.203 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.195+0000 7f11c21b56c0 1 --2- 10.20.193.27:0/478977441 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f11bc060720 0x7f11bc1615b0 unknown :-1 s=CLOSED pgs=246 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:24.203 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.195+0000 7f11c21b56c0 1 -- 10.20.193.27:0/478977441 >> 10.20.193.27:0/478977441 conn(0x7f11bc0ea8b0 msgr2=0x7f11bc10ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:24.203 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.195+0000 7f11c21b56c0 1 -- 10.20.193.27:0/478977441 shutdown_connections 2026-02-13T02:22:24.204 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.195+0000 7f11c21b56c0 1 -- 10.20.193.27:0/478977441 wait complete. 2026-02-13T02:22:24.237 INFO:teuthology.orchestra.run.trial027.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2026-02-13T02:22:22.957076+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":599573,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":115,"num_read_kb":98,"num_write":237,"num_write_kb":3739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":194,"ondisk_log_size":194,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":260,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":8808038400,"kb_used":332580,"kb_used_data":9012,"kb_used_omap":327,"kb_used_meta":323000,"kb_avail":8807705820,"statfs":{"total":9019431321600,"available":9019090759680,"internally_reserved":0,"allocated":9228288,"data_stored":6139082,"data_compressed":114621,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":335075,"internal_metadata":330752797},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9205,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":9,"num_read_kb":7,"num_write":48,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.953679"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016636+0000","last_change":"2026-02-13T02:22:17.023288+0000","last_active":"2026-02-13T02:22:18.016636+0000","last_peered":"2026-02-13T02:22:18.016636+0000","last_clean":"2026-02-13T02:22:18.016636+0000","last_became_active":"2026-02-13T02:22:17.023073+0000","last_became_peered":"2026-02-13T02:22:17.023073+0000","last_unstale":"2026-02-13T02:22:18.016636+0000","last_undegraded":"2026-02-13T02:22:18.016636+0000","last_fullsized":"2026-02-13T02:22:18.016636+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:19:00.322079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047477+0000","last_change":"2026-02-13T02:22:17.018633+0000","last_active":"2026-02-13T02:22:18.047477+0000","last_peered":"2026-02-13T02:22:18.047477+0000","last_clean":"2026-02-13T02:22:18.047477+0000","last_became_active":"2026-02-13T02:22:17.018516+0000","last_became_peered":"2026-02-13T02:22:17.018516+0000","last_unstale":"2026-02-13T02:22:18.047477+0000","last_undegraded":"2026-02-13T02:22:18.047477+0000","last_fullsized":"2026-02-13T02:22:18.047477+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:33:08.892207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047500+0000","last_change":"2026-02-13T02:22:17.019231+0000","last_active":"2026-02-13T02:22:18.047500+0000","last_peered":"2026-02-13T02:22:18.047500+0000","last_clean":"2026-02-13T02:22:18.047500+0000","last_became_active":"2026-02-13T02:22:17.019039+0000","last_became_peered":"2026-02-13T02:22:17.019039+0000","last_unstale":"2026-02-13T02:22:18.047500+0000","last_undegraded":"2026-02-13T02:22:18.047500+0000","last_fullsized":"2026-02-13T02:22:18.047500+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:21:31.573485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2026-02-13T02:22:17.022783+0000","last_change":"2026-02-13T02:22:17.021960+0000","last_active":"2026-02-13T02:22:17.022783+0000","last_peered":"2026-02-13T02:22:17.022783+0000","last_clean":"2026-02-13T02:22:17.022783+0000","last_became_active":"2026-02-13T02:22:17.021325+0000","last_became_peered":"2026-02-13T02:22:17.021325+0000","last_unstale":"2026-02-13T02:22:17.022783+0000","last_undegraded":"2026-02-13T02:22:17.022783+0000","last_fullsized":"2026-02-13T02:22:17.022783+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:56:25.616661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.024818+0000","last_change":"2026-02-13T02:22:17.022417+0000","last_active":"2026-02-13T02:22:18.024818+0000","last_peered":"2026-02-13T02:22:18.024818+0000","last_clean":"2026-02-13T02:22:18.024818+0000","last_became_active":"2026-02-13T02:22:17.022150+0000","last_became_peered":"2026-02-13T02:22:17.022150+0000","last_unstale":"2026-02-13T02:22:18.024818+0000","last_undegraded":"2026-02-13T02:22:18.024818+0000","last_fullsized":"2026-02-13T02:22:18.024818+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T12:26:21.518119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051355+0000","last_change":"2026-02-13T02:22:17.024197+0000","last_active":"2026-02-13T02:22:18.051355+0000","last_peered":"2026-02-13T02:22:18.051355+0000","last_clean":"2026-02-13T02:22:18.051355+0000","last_became_active":"2026-02-13T02:22:17.023378+0000","last_became_peered":"2026-02-13T02:22:17.023378+0000","last_unstale":"2026-02-13T02:22:18.051355+0000","last_undegraded":"2026-02-13T02:22:18.051355+0000","last_fullsized":"2026-02-13T02:22:18.051355+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:17:40.942731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051199+0000","last_change":"2026-02-13T02:22:17.021184+0000","last_active":"2026-02-13T02:22:18.051199+0000","last_peered":"2026-02-13T02:22:18.051199+0000","last_clean":"2026-02-13T02:22:18.051199+0000","last_became_active":"2026-02-13T02:22:17.020961+0000","last_became_peered":"2026-02-13T02:22:17.020961+0000","last_unstale":"2026-02-13T02:22:18.051199+0000","last_undegraded":"2026-02-13T02:22:18.051199+0000","last_fullsized":"2026-02-13T02:22:18.051199+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T10:38:19.326702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.020937+0000","last_change":"2026-02-13T02:22:17.021948+0000","last_active":"2026-02-13T02:22:18.020937+0000","last_peered":"2026-02-13T02:22:18.020937+0000","last_clean":"2026-02-13T02:22:18.020937+0000","last_became_active":"2026-02-13T02:22:17.021760+0000","last_became_peered":"2026-02-13T02:22:17.021760+0000","last_unstale":"2026-02-13T02:22:18.020937+0000","last_undegraded":"2026-02-13T02:22:18.020937+0000","last_fullsized":"2026-02-13T02:22:18.020937+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:24:58.456659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051727+0000","last_change":"2026-02-13T02:22:17.024036+0000","last_active":"2026-02-13T02:22:18.051727+0000","last_peered":"2026-02-13T02:22:18.051727+0000","last_clean":"2026-02-13T02:22:18.051727+0000","last_became_active":"2026-02-13T02:22:17.023805+0000","last_became_peered":"2026-02-13T02:22:17.023805+0000","last_unstale":"2026-02-13T02:22:18.051727+0000","last_undegraded":"2026-02-13T02:22:18.051727+0000","last_fullsized":"2026-02-13T02:22:18.051727+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T02:44:29.738084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051866+0000","last_change":"2026-02-13T02:22:17.024557+0000","last_active":"2026-02-13T02:22:18.051866+0000","last_peered":"2026-02-13T02:22:18.051866+0000","last_clean":"2026-02-13T02:22:18.051866+0000","last_became_active":"2026-02-13T02:22:17.024270+0000","last_became_peered":"2026-02-13T02:22:17.024270+0000","last_unstale":"2026-02-13T02:22:18.051866+0000","last_undegraded":"2026-02-13T02:22:18.051866+0000","last_fullsized":"2026-02-13T02:22:18.051866+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:36:05.815116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051729+0000","last_change":"2026-02-13T02:22:17.023942+0000","last_active":"2026-02-13T02:22:18.051729+0000","last_peered":"2026-02-13T02:22:18.051729+0000","last_clean":"2026-02-13T02:22:18.051729+0000","last_became_active":"2026-02-13T02:22:17.023475+0000","last_became_peered":"2026-02-13T02:22:17.023475+0000","last_unstale":"2026-02-13T02:22:18.051729+0000","last_undegraded":"2026-02-13T02:22:18.051729+0000","last_fullsized":"2026-02-13T02:22:18.051729+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T12:17:44.718186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2026-02-13T02:22:17.023199+0000","last_change":"2026-02-13T02:22:17.022471+0000","last_active":"2026-02-13T02:22:17.023199+0000","last_peered":"2026-02-13T02:22:17.023199+0000","last_clean":"2026-02-13T02:22:17.023199+0000","last_became_active":"2026-02-13T02:22:17.021894+0000","last_became_peered":"2026-02-13T02:22:17.021894+0000","last_unstale":"2026-02-13T02:22:17.023199+0000","last_undegraded":"2026-02-13T02:22:17.023199+0000","last_fullsized":"2026-02-13T02:22:17.023199+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T10:41:27.319466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047429+0000","last_change":"2026-02-13T02:22:17.017830+0000","last_active":"2026-02-13T02:22:18.047429+0000","last_peered":"2026-02-13T02:22:18.047429+0000","last_clean":"2026-02-13T02:22:18.047429+0000","last_became_active":"2026-02-13T02:22:17.017232+0000","last_became_peered":"2026-02-13T02:22:17.017232+0000","last_unstale":"2026-02-13T02:22:18.047429+0000","last_undegraded":"2026-02-13T02:22:18.047429+0000","last_fullsized":"2026-02-13T02:22:18.047429+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:40:47.725661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047355+0000","last_change":"2026-02-13T02:22:17.021764+0000","last_active":"2026-02-13T02:22:18.047355+0000","last_peered":"2026-02-13T02:22:18.047355+0000","last_clean":"2026-02-13T02:22:18.047355+0000","last_became_active":"2026-02-13T02:22:17.021609+0000","last_became_peered":"2026-02-13T02:22:17.021609+0000","last_unstale":"2026-02-13T02:22:18.047355+0000","last_undegraded":"2026-02-13T02:22:18.047355+0000","last_fullsized":"2026-02-13T02:22:18.047355+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T12:04:31.168034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047332+0000","last_change":"2026-02-13T02:22:17.019584+0000","last_active":"2026-02-13T02:22:18.047332+0000","last_peered":"2026-02-13T02:22:18.047332+0000","last_clean":"2026-02-13T02:22:18.047332+0000","last_became_active":"2026-02-13T02:22:17.019388+0000","last_became_peered":"2026-02-13T02:22:17.019388+0000","last_unstale":"2026-02-13T02:22:18.047332+0000","last_undegraded":"2026-02-13T02:22:18.047332+0000","last_fullsized":"2026-02-13T02:22:18.047332+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:33:16.084048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.050998+0000","last_change":"2026-02-13T02:22:17.023950+0000","last_active":"2026-02-13T02:22:18.050998+0000","last_peered":"2026-02-13T02:22:18.050998+0000","last_clean":"2026-02-13T02:22:18.050998+0000","last_became_active":"2026-02-13T02:22:17.023029+0000","last_became_peered":"2026-02-13T02:22:17.023029+0000","last_unstale":"2026-02-13T02:22:18.050998+0000","last_undegraded":"2026-02-13T02:22:18.050998+0000","last_fullsized":"2026-02-13T02:22:18.050998+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:14:25.119381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016670+0000","last_change":"2026-02-13T02:22:17.024675+0000","last_active":"2026-02-13T02:22:18.016670+0000","last_peered":"2026-02-13T02:22:18.016670+0000","last_clean":"2026-02-13T02:22:18.016670+0000","last_became_active":"2026-02-13T02:22:17.024460+0000","last_became_peered":"2026-02-13T02:22:17.024460+0000","last_unstale":"2026-02-13T02:22:18.016670+0000","last_undegraded":"2026-02-13T02:22:18.016670+0000","last_fullsized":"2026-02-13T02:22:18.016670+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T04:41:20.995149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.012018+0000","last_change":"2026-02-13T02:22:17.019261+0000","last_active":"2026-02-13T02:22:18.012018+0000","last_peered":"2026-02-13T02:22:18.012018+0000","last_clean":"2026-02-13T02:22:18.012018+0000","last_became_active":"2026-02-13T02:22:17.019084+0000","last_became_peered":"2026-02-13T02:22:17.019084+0000","last_unstale":"2026-02-13T02:22:18.012018+0000","last_undegraded":"2026-02-13T02:22:18.012018+0000","last_fullsized":"2026-02-13T02:22:18.012018+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:26:06.852915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016186+0000","last_change":"2026-02-13T02:22:17.021336+0000","last_active":"2026-02-13T02:22:18.016186+0000","last_peered":"2026-02-13T02:22:18.016186+0000","last_clean":"2026-02-13T02:22:18.016186+0000","last_became_active":"2026-02-13T02:22:17.020885+0000","last_became_peered":"2026-02-13T02:22:17.020885+0000","last_unstale":"2026-02-13T02:22:18.016186+0000","last_undegraded":"2026-02-13T02:22:18.016186+0000","last_fullsized":"2026-02-13T02:22:18.016186+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T14:05:29.982317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016382+0000","last_change":"2026-02-13T02:22:17.022900+0000","last_active":"2026-02-13T02:22:18.016382+0000","last_peered":"2026-02-13T02:22:18.016382+0000","last_clean":"2026-02-13T02:22:18.016382+0000","last_became_active":"2026-02-13T02:22:17.022753+0000","last_became_peered":"2026-02-13T02:22:17.022753+0000","last_unstale":"2026-02-13T02:22:18.016382+0000","last_undegraded":"2026-02-13T02:22:18.016382+0000","last_fullsized":"2026-02-13T02:22:18.016382+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:35:07.485606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047442+0000","last_change":"2026-02-13T02:22:17.018325+0000","last_active":"2026-02-13T02:22:18.047442+0000","last_peered":"2026-02-13T02:22:18.047442+0000","last_clean":"2026-02-13T02:22:18.047442+0000","last_became_active":"2026-02-13T02:22:17.018167+0000","last_became_peered":"2026-02-13T02:22:17.018167+0000","last_unstale":"2026-02-13T02:22:18.047442+0000","last_undegraded":"2026-02-13T02:22:18.047442+0000","last_fullsized":"2026-02-13T02:22:18.047442+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T14:09:03.876188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051256+0000","last_change":"2026-02-13T02:22:17.022546+0000","last_active":"2026-02-13T02:22:18.051256+0000","last_peered":"2026-02-13T02:22:18.051256+0000","last_clean":"2026-02-13T02:22:18.051256+0000","last_became_active":"2026-02-13T02:22:17.022064+0000","last_became_peered":"2026-02-13T02:22:17.022064+0000","last_unstale":"2026-02-13T02:22:18.051256+0000","last_undegraded":"2026-02-13T02:22:18.051256+0000","last_fullsized":"2026-02-13T02:22:18.051256+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:55:49.327407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.020899+0000","last_change":"2026-02-13T02:22:17.018592+0000","last_active":"2026-02-13T02:22:18.020899+0000","last_peered":"2026-02-13T02:22:18.020899+0000","last_clean":"2026-02-13T02:22:18.020899+0000","last_became_active":"2026-02-13T02:22:17.018368+0000","last_became_peered":"2026-02-13T02:22:17.018368+0000","last_unstale":"2026-02-13T02:22:18.020899+0000","last_undegraded":"2026-02-13T02:22:18.020899+0000","last_fullsized":"2026-02-13T02:22:18.020899+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:15:11.963709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051251+0000","last_change":"2026-02-13T02:22:17.023462+0000","last_active":"2026-02-13T02:22:18.051251+0000","last_peered":"2026-02-13T02:22:18.051251+0000","last_clean":"2026-02-13T02:22:18.051251+0000","last_became_active":"2026-02-13T02:22:17.023327+0000","last_became_peered":"2026-02-13T02:22:17.023327+0000","last_unstale":"2026-02-13T02:22:18.051251+0000","last_undegraded":"2026-02-13T02:22:18.051251+0000","last_fullsized":"2026-02-13T02:22:18.051251+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:42:27.909493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.021321+0000","last_change":"2026-02-13T02:22:17.021211+0000","last_active":"2026-02-13T02:22:18.021321+0000","last_peered":"2026-02-13T02:22:18.021321+0000","last_clean":"2026-02-13T02:22:18.021321+0000","last_became_active":"2026-02-13T02:22:17.021002+0000","last_became_peered":"2026-02-13T02:22:17.021002+0000","last_unstale":"2026-02-13T02:22:18.021321+0000","last_undegraded":"2026-02-13T02:22:18.021321+0000","last_fullsized":"2026-02-13T02:22:18.021321+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:23:00.242280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016321+0000","last_change":"2026-02-13T02:22:17.023595+0000","last_active":"2026-02-13T02:22:18.016321+0000","last_peered":"2026-02-13T02:22:18.016321+0000","last_clean":"2026-02-13T02:22:18.016321+0000","last_became_active":"2026-02-13T02:22:17.023480+0000","last_became_peered":"2026-02-13T02:22:17.023480+0000","last_unstale":"2026-02-13T02:22:18.016321+0000","last_undegraded":"2026-02-13T02:22:18.016321+0000","last_fullsized":"2026-02-13T02:22:18.016321+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:13:14.505114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016555+0000","last_change":"2026-02-13T02:22:17.021599+0000","last_active":"2026-02-13T02:22:18.016555+0000","last_peered":"2026-02-13T02:22:18.016555+0000","last_clean":"2026-02-13T02:22:18.016555+0000","last_became_active":"2026-02-13T02:22:17.021373+0000","last_became_peered":"2026-02-13T02:22:17.021373+0000","last_unstale":"2026-02-13T02:22:18.016555+0000","last_undegraded":"2026-02-13T02:22:18.016555+0000","last_fullsized":"2026-02-13T02:22:18.016555+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:35:13.390476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047383+0000","last_change":"2026-02-13T02:22:17.017507+0000","last_active":"2026-02-13T02:22:18.047383+0000","last_peered":"2026-02-13T02:22:18.047383+0000","last_clean":"2026-02-13T02:22:18.047383+0000","last_became_active":"2026-02-13T02:22:17.017211+0000","last_became_peered":"2026-02-13T02:22:17.017211+0000","last_unstale":"2026-02-13T02:22:18.047383+0000","last_undegraded":"2026-02-13T02:22:18.047383+0000","last_fullsized":"2026-02-13T02:22:18.047383+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:19:19.531222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047561+0000","last_change":"2026-02-13T02:22:17.019486+0000","last_active":"2026-02-13T02:22:18.047561+0000","last_peered":"2026-02-13T02:22:18.047561+0000","last_clean":"2026-02-13T02:22:18.047561+0000","last_became_active":"2026-02-13T02:22:17.019288+0000","last_became_peered":"2026-02-13T02:22:17.019288+0000","last_unstale":"2026-02-13T02:22:18.047561+0000","last_undegraded":"2026-02-13T02:22:18.047561+0000","last_fullsized":"2026-02-13T02:22:18.047561+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:51:14.055654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.015982+0000","last_change":"2026-02-13T02:22:17.022029+0000","last_active":"2026-02-13T02:22:18.015982+0000","last_peered":"2026-02-13T02:22:18.015982+0000","last_clean":"2026-02-13T02:22:18.015982+0000","last_became_active":"2026-02-13T02:22:17.021771+0000","last_became_peered":"2026-02-13T02:22:17.021771+0000","last_unstale":"2026-02-13T02:22:18.015982+0000","last_undegraded":"2026-02-13T02:22:18.015982+0000","last_fullsized":"2026-02-13T02:22:18.015982+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:15:51.936117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016131+0000","last_change":"2026-02-13T02:22:17.021554+0000","last_active":"2026-02-13T02:22:18.016131+0000","last_peered":"2026-02-13T02:22:18.016131+0000","last_clean":"2026-02-13T02:22:18.016131+0000","last_became_active":"2026-02-13T02:22:17.021248+0000","last_became_peered":"2026-02-13T02:22:17.021248+0000","last_unstale":"2026-02-13T02:22:18.016131+0000","last_undegraded":"2026-02-13T02:22:18.016131+0000","last_fullsized":"2026-02-13T02:22:18.016131+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T12:27:26.132715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016224+0000","last_change":"2026-02-13T02:22:17.022426+0000","last_active":"2026-02-13T02:22:18.016224+0000","last_peered":"2026-02-13T02:22:18.016224+0000","last_clean":"2026-02-13T02:22:18.016224+0000","last_became_active":"2026-02-13T02:22:17.022285+0000","last_became_peered":"2026-02-13T02:22:17.022285+0000","last_unstale":"2026-02-13T02:22:18.016224+0000","last_undegraded":"2026-02-13T02:22:18.016224+0000","last_fullsized":"2026-02-13T02:22:18.016224+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:25:58.151252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.024814+0000","last_change":"2026-02-13T02:22:17.022749+0000","last_active":"2026-02-13T02:22:18.024814+0000","last_peered":"2026-02-13T02:22:18.024814+0000","last_clean":"2026-02-13T02:22:18.024814+0000","last_became_active":"2026-02-13T02:22:17.022397+0000","last_became_peered":"2026-02-13T02:22:17.022397+0000","last_unstale":"2026-02-13T02:22:18.024814+0000","last_undegraded":"2026-02-13T02:22:18.024814+0000","last_fullsized":"2026-02-13T02:22:18.024814+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:28:44.768608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"69'1","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051275+0000","last_change":"2026-02-13T02:22:16.014721+0000","last_active":"2026-02-13T02:22:18.051275+0000","last_peered":"2026-02-13T02:22:18.051275+0000","last_clean":"2026-02-13T02:22:18.051275+0000","last_became_active":"2026-02-13T02:22:16.014587+0000","last_became_peered":"2026-02-13T02:22:16.014587+0000","last_unstale":"2026-02-13T02:22:18.051275+0000","last_undegraded":"2026-02-13T02:22:18.051275+0000","last_fullsized":"2026-02-13T02:22:18.051275+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:56:58.003775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051261+0000","last_change":"2026-02-13T02:22:17.022514+0000","last_active":"2026-02-13T02:22:18.051261+0000","last_peered":"2026-02-13T02:22:18.051261+0000","last_clean":"2026-02-13T02:22:18.051261+0000","last_became_active":"2026-02-13T02:22:17.022273+0000","last_became_peered":"2026-02-13T02:22:17.022273+0000","last_unstale":"2026-02-13T02:22:18.051261+0000","last_undegraded":"2026-02-13T02:22:18.051261+0000","last_fullsized":"2026-02-13T02:22:18.051261+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:33:27.805740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"69'1","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051311+0000","last_change":"2026-02-13T02:22:16.012690+0000","last_active":"2026-02-13T02:22:18.051311+0000","last_peered":"2026-02-13T02:22:18.051311+0000","last_clean":"2026-02-13T02:22:18.051311+0000","last_became_active":"2026-02-13T02:22:16.011935+0000","last_became_peered":"2026-02-13T02:22:16.011935+0000","last_unstale":"2026-02-13T02:22:18.051311+0000","last_undegraded":"2026-02-13T02:22:18.051311+0000","last_fullsized":"2026-02-13T02:22:18.051311+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T06:55:46.440839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047121+0000","last_change":"2026-02-13T02:22:16.007738+0000","last_active":"2026-02-13T02:22:18.047121+0000","last_peered":"2026-02-13T02:22:18.047121+0000","last_clean":"2026-02-13T02:22:18.047121+0000","last_became_active":"2026-02-13T02:22:16.007424+0000","last_became_peered":"2026-02-13T02:22:16.007424+0000","last_unstale":"2026-02-13T02:22:18.047121+0000","last_undegraded":"2026-02-13T02:22:18.047121+0000","last_fullsized":"2026-02-13T02:22:18.047121+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:10:41.888980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.020958+0000","last_change":"2026-02-13T02:22:17.017332+0000","last_active":"2026-02-13T02:22:18.020958+0000","last_peered":"2026-02-13T02:22:18.020958+0000","last_clean":"2026-02-13T02:22:18.020958+0000","last_became_active":"2026-02-13T02:22:17.017080+0000","last_became_peered":"2026-02-13T02:22:17.017080+0000","last_unstale":"2026-02-13T02:22:18.020958+0000","last_undegraded":"2026-02-13T02:22:18.020958+0000","last_fullsized":"2026-02-13T02:22:18.020958+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:10:16.349029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051313+0000","last_change":"2026-02-13T02:22:16.014204+0000","last_active":"2026-02-13T02:22:18.051313+0000","last_peered":"2026-02-13T02:22:18.051313+0000","last_clean":"2026-02-13T02:22:18.051313+0000","last_became_active":"2026-02-13T02:22:16.014114+0000","last_became_peered":"2026-02-13T02:22:16.014114+0000","last_unstale":"2026-02-13T02:22:18.051313+0000","last_undegraded":"2026-02-13T02:22:18.051313+0000","last_fullsized":"2026-02-13T02:22:18.051313+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:39:42.557823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.011965+0000","last_change":"2026-02-13T02:22:17.018644+0000","last_active":"2026-02-13T02:22:18.011965+0000","last_peered":"2026-02-13T02:22:18.011965+0000","last_clean":"2026-02-13T02:22:18.011965+0000","last_became_active":"2026-02-13T02:22:17.018511+0000","last_became_peered":"2026-02-13T02:22:17.018511+0000","last_unstale":"2026-02-13T02:22:18.011965+0000","last_undegraded":"2026-02-13T02:22:18.011965+0000","last_fullsized":"2026-02-13T02:22:18.011965+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:18:39.510344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051718+0000","last_change":"2026-02-13T02:22:16.013123+0000","last_active":"2026-02-13T02:22:18.051718+0000","last_peered":"2026-02-13T02:22:18.051718+0000","last_clean":"2026-02-13T02:22:18.051718+0000","last_became_active":"2026-02-13T02:22:16.012986+0000","last_became_peered":"2026-02-13T02:22:16.012986+0000","last_unstale":"2026-02-13T02:22:18.051718+0000","last_undegraded":"2026-02-13T02:22:18.051718+0000","last_fullsized":"2026-02-13T02:22:18.051718+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T10:09:29.447645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051655+0000","last_change":"2026-02-13T02:22:17.024718+0000","last_active":"2026-02-13T02:22:18.051655+0000","last_peered":"2026-02-13T02:22:18.051655+0000","last_clean":"2026-02-13T02:22:18.051655+0000","last_became_active":"2026-02-13T02:22:17.024333+0000","last_became_peered":"2026-02-13T02:22:17.024333+0000","last_unstale":"2026-02-13T02:22:18.051655+0000","last_undegraded":"2026-02-13T02:22:18.051655+0000","last_fullsized":"2026-02-13T02:22:18.051655+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:44:11.055118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051693+0000","last_change":"2026-02-13T02:22:16.015080+0000","last_active":"2026-02-13T02:22:18.051693+0000","last_peered":"2026-02-13T02:22:18.051693+0000","last_clean":"2026-02-13T02:22:18.051693+0000","last_became_active":"2026-02-13T02:22:16.014694+0000","last_became_peered":"2026-02-13T02:22:16.014694+0000","last_unstale":"2026-02-13T02:22:18.051693+0000","last_undegraded":"2026-02-13T02:22:18.051693+0000","last_fullsized":"2026-02-13T02:22:18.051693+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T06:02:28.163616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047603+0000","last_change":"2026-02-13T02:22:17.019516+0000","last_active":"2026-02-13T02:22:18.047603+0000","last_peered":"2026-02-13T02:22:18.047603+0000","last_clean":"2026-02-13T02:22:18.047603+0000","last_became_active":"2026-02-13T02:22:17.019234+0000","last_became_peered":"2026-02-13T02:22:17.019234+0000","last_unstale":"2026-02-13T02:22:18.047603+0000","last_undegraded":"2026-02-13T02:22:18.047603+0000","last_fullsized":"2026-02-13T02:22:18.047603+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T10:25:31.467398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"69'3","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051378+0000","last_change":"2026-02-13T02:22:16.013733+0000","last_active":"2026-02-13T02:22:18.051378+0000","last_peered":"2026-02-13T02:22:18.051378+0000","last_clean":"2026-02-13T02:22:18.051378+0000","last_became_active":"2026-02-13T02:22:16.013472+0000","last_became_peered":"2026-02-13T02:22:16.013472+0000","last_unstale":"2026-02-13T02:22:18.051378+0000","last_undegraded":"2026-02-13T02:22:18.051378+0000","last_fullsized":"2026-02-13T02:22:18.051378+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:59:36.928070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051362+0000","last_change":"2026-02-13T02:22:17.023907+0000","last_active":"2026-02-13T02:22:18.051362+0000","last_peered":"2026-02-13T02:22:18.051362+0000","last_clean":"2026-02-13T02:22:18.051362+0000","last_became_active":"2026-02-13T02:22:17.022420+0000","last_became_peered":"2026-02-13T02:22:17.022420+0000","last_unstale":"2026-02-13T02:22:18.051362+0000","last_undegraded":"2026-02-13T02:22:18.051362+0000","last_fullsized":"2026-02-13T02:22:18.051362+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:29:25.440291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"70'9","reported_seq":31,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:21.436149+0000","last_change":"2026-02-13T02:22:16.013888+0000","last_active":"2026-02-13T02:22:21.436149+0000","last_peered":"2026-02-13T02:22:21.436149+0000","last_clean":"2026-02-13T02:22:21.436149+0000","last_became_active":"2026-02-13T02:22:16.013652+0000","last_became_peered":"2026-02-13T02:22:16.013652+0000","last_unstale":"2026-02-13T02:22:21.436149+0000","last_undegraded":"2026-02-13T02:22:21.436149+0000","last_fullsized":"2026-02-13T02:22:21.436149+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:25:28.136946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8385,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":9,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.025043+0000","last_change":"2026-02-13T02:22:17.023547+0000","last_active":"2026-02-13T02:22:18.025043+0000","last_peered":"2026-02-13T02:22:18.025043+0000","last_clean":"2026-02-13T02:22:18.025043+0000","last_became_active":"2026-02-13T02:22:17.023410+0000","last_became_peered":"2026-02-13T02:22:17.023410+0000","last_unstale":"2026-02-13T02:22:18.025043+0000","last_undegraded":"2026-02-13T02:22:18.025043+0000","last_fullsized":"2026-02-13T02:22:18.025043+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T04:35:03.522242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"69'2","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.015919+0000","last_change":"2026-02-13T02:22:16.014701+0000","last_active":"2026-02-13T02:22:18.015919+0000","last_peered":"2026-02-13T02:22:18.015919+0000","last_clean":"2026-02-13T02:22:18.015919+0000","last_became_active":"2026-02-13T02:22:16.014476+0000","last_became_peered":"2026-02-13T02:22:16.014476+0000","last_unstale":"2026-02-13T02:22:18.015919+0000","last_undegraded":"2026-02-13T02:22:18.015919+0000","last_fullsized":"2026-02-13T02:22:18.015919+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:25:47.865783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047309+0000","last_change":"2026-02-13T02:22:17.017924+0000","last_active":"2026-02-13T02:22:18.047309+0000","last_peered":"2026-02-13T02:22:18.047309+0000","last_clean":"2026-02-13T02:22:18.047309+0000","last_became_active":"2026-02-13T02:22:17.017666+0000","last_became_peered":"2026-02-13T02:22:17.017666+0000","last_unstale":"2026-02-13T02:22:18.047309+0000","last_undegraded":"2026-02-13T02:22:18.047309+0000","last_fullsized":"2026-02-13T02:22:18.047309+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:38:38.063114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051219+0000","last_change":"2026-02-13T02:22:16.014437+0000","last_active":"2026-02-13T02:22:18.051219+0000","last_peered":"2026-02-13T02:22:18.051219+0000","last_clean":"2026-02-13T02:22:18.051219+0000","last_became_active":"2026-02-13T02:22:16.014293+0000","last_became_peered":"2026-02-13T02:22:16.014293+0000","last_unstale":"2026-02-13T02:22:18.051219+0000","last_undegraded":"2026-02-13T02:22:18.051219+0000","last_fullsized":"2026-02-13T02:22:18.051219+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:08:58.340775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051558+0000","last_change":"2026-02-13T02:22:17.023621+0000","last_active":"2026-02-13T02:22:18.051558+0000","last_peered":"2026-02-13T02:22:18.051558+0000","last_clean":"2026-02-13T02:22:18.051558+0000","last_became_active":"2026-02-13T02:22:17.023343+0000","last_became_peered":"2026-02-13T02:22:17.023343+0000","last_unstale":"2026-02-13T02:22:18.051558+0000","last_undegraded":"2026-02-13T02:22:18.051558+0000","last_fullsized":"2026-02-13T02:22:18.051558+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T06:36:08.489377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2026-02-13T02:22:17.015247+0000","last_change":"2026-02-13T02:22:16.013825+0000","last_active":"2026-02-13T02:22:17.015247+0000","last_peered":"2026-02-13T02:22:17.015247+0000","last_clean":"2026-02-13T02:22:17.015247+0000","last_became_active":"2026-02-13T02:22:16.013704+0000","last_became_peered":"2026-02-13T02:22:16.013704+0000","last_unstale":"2026-02-13T02:22:17.015247+0000","last_undegraded":"2026-02-13T02:22:17.015247+0000","last_fullsized":"2026-02-13T02:22:17.015247+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T02:40:53.106423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051148+0000","last_change":"2026-02-13T02:22:17.022293+0000","last_active":"2026-02-13T02:22:18.051148+0000","last_peered":"2026-02-13T02:22:18.051148+0000","last_clean":"2026-02-13T02:22:18.051148+0000","last_became_active":"2026-02-13T02:22:17.022130+0000","last_became_peered":"2026-02-13T02:22:17.022130+0000","last_unstale":"2026-02-13T02:22:18.051148+0000","last_undegraded":"2026-02-13T02:22:18.051148+0000","last_fullsized":"2026-02-13T02:22:18.051148+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:33:03.123178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"69'8","reported_seq":24,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.024641+0000","last_change":"2026-02-13T02:22:16.013612+0000","last_active":"2026-02-13T02:22:18.024641+0000","last_peered":"2026-02-13T02:22:18.024641+0000","last_clean":"2026-02-13T02:22:18.024641+0000","last_became_active":"2026-02-13T02:22:16.013444+0000","last_became_peered":"2026-02-13T02:22:16.013444+0000","last_unstale":"2026-02-13T02:22:18.024641+0000","last_undegraded":"2026-02-13T02:22:18.024641+0000","last_fullsized":"2026-02-13T02:22:18.024641+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T04:59:32.263657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051261+0000","last_change":"2026-02-13T02:22:17.024692+0000","last_active":"2026-02-13T02:22:18.051261+0000","last_peered":"2026-02-13T02:22:18.051261+0000","last_clean":"2026-02-13T02:22:18.051261+0000","last_became_active":"2026-02-13T02:22:17.024543+0000","last_became_peered":"2026-02-13T02:22:17.024543+0000","last_unstale":"2026-02-13T02:22:18.051261+0000","last_undegraded":"2026-02-13T02:22:18.051261+0000","last_fullsized":"2026-02-13T02:22:18.051261+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:04:25.726793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"69'1","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.025073+0000","last_change":"2026-02-13T02:22:16.011678+0000","last_active":"2026-02-13T02:22:18.025073+0000","last_peered":"2026-02-13T02:22:18.025073+0000","last_clean":"2026-02-13T02:22:18.025073+0000","last_became_active":"2026-02-13T02:22:16.010914+0000","last_became_peered":"2026-02-13T02:22:16.010914+0000","last_unstale":"2026-02-13T02:22:18.025073+0000","last_undegraded":"2026-02-13T02:22:18.025073+0000","last_fullsized":"2026-02-13T02:22:18.025073+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:44:29.583923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016496+0000","last_change":"2026-02-13T02:22:17.023312+0000","last_active":"2026-02-13T02:22:18.016496+0000","last_peered":"2026-02-13T02:22:18.016496+0000","last_clean":"2026-02-13T02:22:18.016496+0000","last_became_active":"2026-02-13T02:22:17.023084+0000","last_became_peered":"2026-02-13T02:22:17.023084+0000","last_unstale":"2026-02-13T02:22:18.016496+0000","last_undegraded":"2026-02-13T02:22:18.016496+0000","last_fullsized":"2026-02-13T02:22:18.016496+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T10:27:31.848680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"69'2","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047389+0000","last_change":"2026-02-13T02:22:16.010281+0000","last_active":"2026-02-13T02:22:18.047389+0000","last_peered":"2026-02-13T02:22:18.047389+0000","last_clean":"2026-02-13T02:22:18.047389+0000","last_became_active":"2026-02-13T02:22:16.010075+0000","last_became_peered":"2026-02-13T02:22:16.010075+0000","last_unstale":"2026-02-13T02:22:18.047389+0000","last_undegraded":"2026-02-13T02:22:18.047389+0000","last_fullsized":"2026-02-13T02:22:18.047389+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:25:52.223102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051820+0000","last_change":"2026-02-13T02:22:17.023958+0000","last_active":"2026-02-13T02:22:18.051820+0000","last_peered":"2026-02-13T02:22:18.051820+0000","last_clean":"2026-02-13T02:22:18.051820+0000","last_became_active":"2026-02-13T02:22:17.023529+0000","last_became_peered":"2026-02-13T02:22:17.023529+0000","last_unstale":"2026-02-13T02:22:18.051820+0000","last_undegraded":"2026-02-13T02:22:18.051820+0000","last_fullsized":"2026-02-13T02:22:18.051820+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T12:45:26.937375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"69'1","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051446+0000","last_change":"2026-02-13T02:22:16.013663+0000","last_active":"2026-02-13T02:22:18.051446+0000","last_peered":"2026-02-13T02:22:18.051446+0000","last_clean":"2026-02-13T02:22:18.051446+0000","last_became_active":"2026-02-13T02:22:16.013445+0000","last_became_peered":"2026-02-13T02:22:16.013445+0000","last_unstale":"2026-02-13T02:22:18.051446+0000","last_undegraded":"2026-02-13T02:22:18.051446+0000","last_fullsized":"2026-02-13T02:22:18.051446+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:02:58.149380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051679+0000","last_change":"2026-02-13T02:22:17.021199+0000","last_active":"2026-02-13T02:22:18.051679+0000","last_peered":"2026-02-13T02:22:18.051679+0000","last_clean":"2026-02-13T02:22:18.051679+0000","last_became_active":"2026-02-13T02:22:17.021008+0000","last_became_peered":"2026-02-13T02:22:17.021008+0000","last_unstale":"2026-02-13T02:22:18.051679+0000","last_undegraded":"2026-02-13T02:22:18.051679+0000","last_fullsized":"2026-02-13T02:22:18.051679+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:21:08.010400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016275+0000","last_change":"2026-02-13T02:22:16.017721+0000","last_active":"2026-02-13T02:22:18.016275+0000","last_peered":"2026-02-13T02:22:18.016275+0000","last_clean":"2026-02-13T02:22:18.016275+0000","last_became_active":"2026-02-13T02:22:16.017552+0000","last_became_peered":"2026-02-13T02:22:16.017552+0000","last_unstale":"2026-02-13T02:22:18.016275+0000","last_undegraded":"2026-02-13T02:22:18.016275+0000","last_fullsized":"2026-02-13T02:22:18.016275+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:14:18.034709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051822+0000","last_change":"2026-02-13T02:22:17.025362+0000","last_active":"2026-02-13T02:22:18.051822+0000","last_peered":"2026-02-13T02:22:18.051822+0000","last_clean":"2026-02-13T02:22:18.051822+0000","last_became_active":"2026-02-13T02:22:17.024942+0000","last_became_peered":"2026-02-13T02:22:17.024942+0000","last_unstale":"2026-02-13T02:22:18.051822+0000","last_undegraded":"2026-02-13T02:22:18.051822+0000","last_fullsized":"2026-02-13T02:22:18.051822+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:09:37.454791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'152","reported_seq":339,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051251+0000","last_change":"2026-02-13T02:21:55.384973+0000","last_active":"2026-02-13T02:22:18.051251+0000","last_peered":"2026-02-13T02:22:18.051251+0000","last_clean":"2026-02-13T02:22:18.051251+0000","last_became_active":"2026-02-13T02:21:55.384610+0000","last_became_peered":"2026-02-13T02:21:55.384610+0000","last_unstale":"2026-02-13T02:22:18.051251+0000","last_undegraded":"2026-02-13T02:22:18.051251+0000","last_fullsized":"2026-02-13T02:22:18.051251+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:20:55.621197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:20:55.621197+0000","last_clean_scrub_stamp":"2026-02-13T02:20:55.621197+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:45:14.548172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":189,"num_write_kb":3716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2026-02-13T02:22:17.015114+0000","last_change":"2026-02-13T02:22:16.012172+0000","last_active":"2026-02-13T02:22:17.015114+0000","last_peered":"2026-02-13T02:22:17.015114+0000","last_clean":"2026-02-13T02:22:17.015114+0000","last_became_active":"2026-02-13T02:22:16.011646+0000","last_became_peered":"2026-02-13T02:22:16.011646+0000","last_unstale":"2026-02-13T02:22:17.015114+0000","last_undegraded":"2026-02-13T02:22:17.015114+0000","last_fullsized":"2026-02-13T02:22:17.015114+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:05:01.007415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.021079+0000","last_change":"2026-02-13T02:22:17.020741+0000","last_active":"2026-02-13T02:22:18.021079+0000","last_peered":"2026-02-13T02:22:18.021079+0000","last_clean":"2026-02-13T02:22:18.021079+0000","last_became_active":"2026-02-13T02:22:17.020574+0000","last_became_peered":"2026-02-13T02:22:17.020574+0000","last_unstale":"2026-02-13T02:22:18.021079+0000","last_undegraded":"2026-02-13T02:22:18.021079+0000","last_fullsized":"2026-02-13T02:22:18.021079+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T06:28:11.279493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051246+0000","last_change":"2026-02-13T02:22:16.014565+0000","last_active":"2026-02-13T02:22:18.051246+0000","last_peered":"2026-02-13T02:22:18.051246+0000","last_clean":"2026-02-13T02:22:18.051246+0000","last_became_active":"2026-02-13T02:22:16.013815+0000","last_became_peered":"2026-02-13T02:22:16.013815+0000","last_unstale":"2026-02-13T02:22:18.051246+0000","last_undegraded":"2026-02-13T02:22:18.051246+0000","last_fullsized":"2026-02-13T02:22:18.051246+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T04:34:31.290868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047168+0000","last_change":"2026-02-13T02:22:17.013289+0000","last_active":"2026-02-13T02:22:18.047168+0000","last_peered":"2026-02-13T02:22:18.047168+0000","last_clean":"2026-02-13T02:22:18.047168+0000","last_became_active":"2026-02-13T02:22:17.013160+0000","last_became_peered":"2026-02-13T02:22:17.013160+0000","last_unstale":"2026-02-13T02:22:18.047168+0000","last_undegraded":"2026-02-13T02:22:18.047168+0000","last_fullsized":"2026-02-13T02:22:18.047168+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:19:06.942113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047197+0000","last_change":"2026-02-13T02:22:16.007420+0000","last_active":"2026-02-13T02:22:18.047197+0000","last_peered":"2026-02-13T02:22:18.047197+0000","last_clean":"2026-02-13T02:22:18.047197+0000","last_became_active":"2026-02-13T02:22:16.007114+0000","last_became_peered":"2026-02-13T02:22:16.007114+0000","last_unstale":"2026-02-13T02:22:18.047197+0000","last_undegraded":"2026-02-13T02:22:18.047197+0000","last_fullsized":"2026-02-13T02:22:18.047197+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:54:40.155679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051284+0000","last_change":"2026-02-13T02:22:17.021679+0000","last_active":"2026-02-13T02:22:18.051284+0000","last_peered":"2026-02-13T02:22:18.051284+0000","last_clean":"2026-02-13T02:22:18.051284+0000","last_became_active":"2026-02-13T02:22:17.021041+0000","last_became_peered":"2026-02-13T02:22:17.021041+0000","last_unstale":"2026-02-13T02:22:18.051284+0000","last_undegraded":"2026-02-13T02:22:18.051284+0000","last_fullsized":"2026-02-13T02:22:18.051284+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T04:37:08.611062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051417+0000","last_change":"2026-02-13T02:22:16.016749+0000","last_active":"2026-02-13T02:22:18.051417+0000","last_peered":"2026-02-13T02:22:18.051417+0000","last_clean":"2026-02-13T02:22:18.051417+0000","last_became_active":"2026-02-13T02:22:16.016620+0000","last_became_peered":"2026-02-13T02:22:16.016620+0000","last_unstale":"2026-02-13T02:22:18.051417+0000","last_undegraded":"2026-02-13T02:22:18.051417+0000","last_fullsized":"2026-02-13T02:22:18.051417+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:29:12.370221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051646+0000","last_change":"2026-02-13T02:22:17.025708+0000","last_active":"2026-02-13T02:22:18.051646+0000","last_peered":"2026-02-13T02:22:18.051646+0000","last_clean":"2026-02-13T02:22:18.051646+0000","last_became_active":"2026-02-13T02:22:17.025240+0000","last_became_peered":"2026-02-13T02:22:17.025240+0000","last_unstale":"2026-02-13T02:22:18.051646+0000","last_undegraded":"2026-02-13T02:22:18.051646+0000","last_fullsized":"2026-02-13T02:22:18.051646+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:04:02.085370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.011935+0000","last_change":"2026-02-13T02:22:16.009592+0000","last_active":"2026-02-13T02:22:18.011935+0000","last_peered":"2026-02-13T02:22:18.011935+0000","last_clean":"2026-02-13T02:22:18.011935+0000","last_became_active":"2026-02-13T02:22:16.009465+0000","last_became_peered":"2026-02-13T02:22:16.009465+0000","last_unstale":"2026-02-13T02:22:18.011935+0000","last_undegraded":"2026-02-13T02:22:18.011935+0000","last_fullsized":"2026-02-13T02:22:18.011935+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T02:33:29.133373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051794+0000","last_change":"2026-02-13T02:22:17.023729+0000","last_active":"2026-02-13T02:22:18.051794+0000","last_peered":"2026-02-13T02:22:18.051794+0000","last_clean":"2026-02-13T02:22:18.051794+0000","last_became_active":"2026-02-13T02:22:17.022007+0000","last_became_peered":"2026-02-13T02:22:17.022007+0000","last_unstale":"2026-02-13T02:22:18.051794+0000","last_undegraded":"2026-02-13T02:22:18.051794+0000","last_fullsized":"2026-02-13T02:22:18.051794+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T10:34:43.688805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.025077+0000","last_change":"2026-02-13T02:22:16.012023+0000","last_active":"2026-02-13T02:22:18.025077+0000","last_peered":"2026-02-13T02:22:18.025077+0000","last_clean":"2026-02-13T02:22:18.025077+0000","last_became_active":"2026-02-13T02:22:16.011680+0000","last_became_peered":"2026-02-13T02:22:16.011680+0000","last_unstale":"2026-02-13T02:22:18.025077+0000","last_undegraded":"2026-02-13T02:22:18.025077+0000","last_fullsized":"2026-02-13T02:22:18.025077+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T02:28:59.078471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047450+0000","last_change":"2026-02-13T02:22:17.015877+0000","last_active":"2026-02-13T02:22:18.047450+0000","last_peered":"2026-02-13T02:22:18.047450+0000","last_clean":"2026-02-13T02:22:18.047450+0000","last_became_active":"2026-02-13T02:22:17.015701+0000","last_became_peered":"2026-02-13T02:22:17.015701+0000","last_unstale":"2026-02-13T02:22:18.047450+0000","last_undegraded":"2026-02-13T02:22:18.047450+0000","last_fullsized":"2026-02-13T02:22:18.047450+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:35:45.903754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"70'1","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.020452+0000","last_change":"2026-02-13T02:22:16.008749+0000","last_active":"2026-02-13T02:22:18.020452+0000","last_peered":"2026-02-13T02:22:18.020452+0000","last_clean":"2026-02-13T02:22:18.020452+0000","last_became_active":"2026-02-13T02:22:16.008597+0000","last_became_peered":"2026-02-13T02:22:16.008597+0000","last_unstale":"2026-02-13T02:22:18.020452+0000","last_undegraded":"2026-02-13T02:22:18.020452+0000","last_fullsized":"2026-02-13T02:22:18.020452+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T10:18:25.963625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.021323+0000","last_change":"2026-02-13T02:22:17.017797+0000","last_active":"2026-02-13T02:22:18.021323+0000","last_peered":"2026-02-13T02:22:18.021323+0000","last_clean":"2026-02-13T02:22:18.021323+0000","last_became_active":"2026-02-13T02:22:17.017174+0000","last_became_peered":"2026-02-13T02:22:17.017174+0000","last_unstale":"2026-02-13T02:22:18.021323+0000","last_undegraded":"2026-02-13T02:22:18.021323+0000","last_fullsized":"2026-02-13T02:22:18.021323+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:39:41.102807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"69'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2026-02-13T02:22:17.026121+0000","last_change":"2026-02-13T02:22:16.017646+0000","last_active":"2026-02-13T02:22:17.026121+0000","last_peered":"2026-02-13T02:22:17.026121+0000","last_clean":"2026-02-13T02:22:17.026121+0000","last_became_active":"2026-02-13T02:22:16.017529+0000","last_became_peered":"2026-02-13T02:22:16.017529+0000","last_unstale":"2026-02-13T02:22:17.026121+0000","last_undegraded":"2026-02-13T02:22:17.026121+0000","last_fullsized":"2026-02-13T02:22:17.026121+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T12:34:48.977200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047594+0000","last_change":"2026-02-13T02:22:17.019503+0000","last_active":"2026-02-13T02:22:18.047594+0000","last_peered":"2026-02-13T02:22:18.047594+0000","last_clean":"2026-02-13T02:22:18.047594+0000","last_became_active":"2026-02-13T02:22:17.019259+0000","last_became_peered":"2026-02-13T02:22:17.019259+0000","last_unstale":"2026-02-13T02:22:18.047594+0000","last_undegraded":"2026-02-13T02:22:18.047594+0000","last_fullsized":"2026-02-13T02:22:18.047594+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:19:49.339245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.050971+0000","last_change":"2026-02-13T02:22:16.016248+0000","last_active":"2026-02-13T02:22:18.050971+0000","last_peered":"2026-02-13T02:22:18.050971+0000","last_clean":"2026-02-13T02:22:18.050971+0000","last_became_active":"2026-02-13T02:22:16.016016+0000","last_became_peered":"2026-02-13T02:22:16.016016+0000","last_unstale":"2026-02-13T02:22:18.050971+0000","last_undegraded":"2026-02-13T02:22:18.050971+0000","last_fullsized":"2026-02-13T02:22:18.050971+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:39:11.300311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.024643+0000","last_change":"2026-02-13T02:22:17.023656+0000","last_active":"2026-02-13T02:22:18.024643+0000","last_peered":"2026-02-13T02:22:18.024643+0000","last_clean":"2026-02-13T02:22:18.024643+0000","last_became_active":"2026-02-13T02:22:17.023524+0000","last_became_peered":"2026-02-13T02:22:17.023524+0000","last_unstale":"2026-02-13T02:22:18.024643+0000","last_undegraded":"2026-02-13T02:22:18.024643+0000","last_fullsized":"2026-02-13T02:22:18.024643+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:33:34.390446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.020934+0000","last_change":"2026-02-13T02:22:16.010000+0000","last_active":"2026-02-13T02:22:18.020934+0000","last_peered":"2026-02-13T02:22:18.020934+0000","last_clean":"2026-02-13T02:22:18.020934+0000","last_became_active":"2026-02-13T02:22:16.009736+0000","last_became_peered":"2026-02-13T02:22:16.009736+0000","last_unstale":"2026-02-13T02:22:18.020934+0000","last_undegraded":"2026-02-13T02:22:18.020934+0000","last_fullsized":"2026-02-13T02:22:18.020934+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:16:46.363188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016302+0000","last_change":"2026-02-13T02:22:17.022375+0000","last_active":"2026-02-13T02:22:18.016302+0000","last_peered":"2026-02-13T02:22:18.016302+0000","last_clean":"2026-02-13T02:22:18.016302+0000","last_became_active":"2026-02-13T02:22:17.022236+0000","last_became_peered":"2026-02-13T02:22:17.022236+0000","last_unstale":"2026-02-13T02:22:18.016302+0000","last_undegraded":"2026-02-13T02:22:18.016302+0000","last_fullsized":"2026-02-13T02:22:18.016302+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:32:39.302116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.015763+0000","last_change":"2026-02-13T02:22:16.015222+0000","last_active":"2026-02-13T02:22:18.015763+0000","last_peered":"2026-02-13T02:22:18.015763+0000","last_clean":"2026-02-13T02:22:18.015763+0000","last_became_active":"2026-02-13T02:22:16.014947+0000","last_became_peered":"2026-02-13T02:22:16.014947+0000","last_unstale":"2026-02-13T02:22:18.015763+0000","last_undegraded":"2026-02-13T02:22:18.015763+0000","last_fullsized":"2026-02-13T02:22:18.015763+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T14:01:02.677585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.021118+0000","last_change":"2026-02-13T02:22:17.021948+0000","last_active":"2026-02-13T02:22:18.021118+0000","last_peered":"2026-02-13T02:22:18.021118+0000","last_clean":"2026-02-13T02:22:18.021118+0000","last_became_active":"2026-02-13T02:22:17.021764+0000","last_became_peered":"2026-02-13T02:22:17.021764+0000","last_unstale":"2026-02-13T02:22:18.021118+0000","last_undegraded":"2026-02-13T02:22:18.021118+0000","last_fullsized":"2026-02-13T02:22:18.021118+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:39:50.788814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"69'2","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.024813+0000","last_change":"2026-02-13T02:22:16.014740+0000","last_active":"2026-02-13T02:22:18.024813+0000","last_peered":"2026-02-13T02:22:18.024813+0000","last_clean":"2026-02-13T02:22:18.024813+0000","last_became_active":"2026-02-13T02:22:16.014578+0000","last_became_peered":"2026-02-13T02:22:16.014578+0000","last_unstale":"2026-02-13T02:22:18.024813+0000","last_undegraded":"2026-02-13T02:22:18.024813+0000","last_fullsized":"2026-02-13T02:22:18.024813+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:09:10.619351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051553+0000","last_change":"2026-02-13T02:22:17.025211+0000","last_active":"2026-02-13T02:22:18.051553+0000","last_peered":"2026-02-13T02:22:18.051553+0000","last_clean":"2026-02-13T02:22:18.051553+0000","last_became_active":"2026-02-13T02:22:17.025015+0000","last_became_peered":"2026-02-13T02:22:17.025015+0000","last_unstale":"2026-02-13T02:22:18.051553+0000","last_undegraded":"2026-02-13T02:22:18.051553+0000","last_fullsized":"2026-02-13T02:22:18.051553+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T04:28:08.456016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051394+0000","last_change":"2026-02-13T02:22:17.022434+0000","last_active":"2026-02-13T02:22:18.051394+0000","last_peered":"2026-02-13T02:22:18.051394+0000","last_clean":"2026-02-13T02:22:18.051394+0000","last_became_active":"2026-02-13T02:22:17.022160+0000","last_became_peered":"2026-02-13T02:22:17.022160+0000","last_unstale":"2026-02-13T02:22:18.051394+0000","last_undegraded":"2026-02-13T02:22:18.051394+0000","last_fullsized":"2026-02-13T02:22:18.051394+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T12:27:17.994699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"69'2","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.024901+0000","last_change":"2026-02-13T02:22:16.013867+0000","last_active":"2026-02-13T02:22:18.024901+0000","last_peered":"2026-02-13T02:22:18.024901+0000","last_clean":"2026-02-13T02:22:18.024901+0000","last_became_active":"2026-02-13T02:22:16.013689+0000","last_became_peered":"2026-02-13T02:22:16.013689+0000","last_unstale":"2026-02-13T02:22:18.024901+0000","last_undegraded":"2026-02-13T02:22:18.024901+0000","last_fullsized":"2026-02-13T02:22:18.024901+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T04:46:39.476015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051182+0000","last_change":"2026-02-13T02:22:17.022407+0000","last_active":"2026-02-13T02:22:18.051182+0000","last_peered":"2026-02-13T02:22:18.051182+0000","last_clean":"2026-02-13T02:22:18.051182+0000","last_became_active":"2026-02-13T02:22:17.021354+0000","last_became_peered":"2026-02-13T02:22:17.021354+0000","last_unstale":"2026-02-13T02:22:18.051182+0000","last_undegraded":"2026-02-13T02:22:18.051182+0000","last_fullsized":"2026-02-13T02:22:18.051182+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:35:18.694829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"69'2","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.025017+0000","last_change":"2026-02-13T02:22:16.015538+0000","last_active":"2026-02-13T02:22:18.025017+0000","last_peered":"2026-02-13T02:22:18.025017+0000","last_clean":"2026-02-13T02:22:18.025017+0000","last_became_active":"2026-02-13T02:22:16.015441+0000","last_became_peered":"2026-02-13T02:22:16.015441+0000","last_unstale":"2026-02-13T02:22:18.025017+0000","last_undegraded":"2026-02-13T02:22:18.025017+0000","last_fullsized":"2026-02-13T02:22:18.025017+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T02:34:39.849666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047574+0000","last_change":"2026-02-13T02:22:17.018483+0000","last_active":"2026-02-13T02:22:18.047574+0000","last_peered":"2026-02-13T02:22:18.047574+0000","last_clean":"2026-02-13T02:22:18.047574+0000","last_became_active":"2026-02-13T02:22:17.018318+0000","last_became_peered":"2026-02-13T02:22:17.018318+0000","last_unstale":"2026-02-13T02:22:18.047574+0000","last_undegraded":"2026-02-13T02:22:18.047574+0000","last_fullsized":"2026-02-13T02:22:18.047574+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T10:22:49.603621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"69'1","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.021277+0000","last_change":"2026-02-13T02:22:16.008795+0000","last_active":"2026-02-13T02:22:18.021277+0000","last_peered":"2026-02-13T02:22:18.021277+0000","last_clean":"2026-02-13T02:22:18.021277+0000","last_became_active":"2026-02-13T02:22:16.008621+0000","last_became_peered":"2026-02-13T02:22:16.008621+0000","last_unstale":"2026-02-13T02:22:18.021277+0000","last_undegraded":"2026-02-13T02:22:18.021277+0000","last_fullsized":"2026-02-13T02:22:18.021277+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:43:45.861077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.021240+0000","last_change":"2026-02-13T02:22:17.019826+0000","last_active":"2026-02-13T02:22:18.021240+0000","last_peered":"2026-02-13T02:22:18.021240+0000","last_clean":"2026-02-13T02:22:18.021240+0000","last_became_active":"2026-02-13T02:22:17.019429+0000","last_became_peered":"2026-02-13T02:22:17.019429+0000","last_unstale":"2026-02-13T02:22:18.021240+0000","last_undegraded":"2026-02-13T02:22:18.021240+0000","last_fullsized":"2026-02-13T02:22:18.021240+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:52:59.597088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"69'4","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.012046+0000","last_change":"2026-02-13T02:22:16.011002+0000","last_active":"2026-02-13T02:22:18.012046+0000","last_peered":"2026-02-13T02:22:18.012046+0000","last_clean":"2026-02-13T02:22:18.012046+0000","last_became_active":"2026-02-13T02:22:16.010892+0000","last_became_peered":"2026-02-13T02:22:16.010892+0000","last_unstale":"2026-02-13T02:22:18.012046+0000","last_undegraded":"2026-02-13T02:22:18.012046+0000","last_fullsized":"2026-02-13T02:22:18.012046+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:57:22.333777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":604,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":9205,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":9,"num_read_kb":7,"num_write":48,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":16582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":65369,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":189,"num_write_kb":3716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":264161,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":65,"seq":279172874245,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27696,"kb_used_data":732,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":733975504,"statfs":{"total":751619276800,"available":751590916096,"internally_reserved":0,"allocated":749568,"data_stored":364081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8466,"internal_metadata":27582190},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":58,"seq":249108103175,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27768,"kb_used_data":804,"kb_used_omap":18,"kb_used_meta":26925,"kb_avail":733975432,"statfs":{"total":751619276800,"available":751590842368,"internally_reserved":0,"allocated":823296,"data_stored":956277,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":19356,"internal_metadata":27571300},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":54,"seq":231928233993,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27700,"kb_used_data":736,"kb_used_omap":12,"kb_used_meta":26931,"kb_avail":733975500,"statfs":{"total":751619276800,"available":751590912000,"internally_reserved":0,"allocated":753664,"data_stored":364685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12485,"internal_metadata":27578171},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":47,"seq":201863462922,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27704,"kb_used_data":740,"kb_used_omap":20,"kb_used_meta":26923,"kb_avail":733975496,"statfs":{"total":751619276800,"available":751590907904,"internally_reserved":0,"allocated":757760,"data_stored":364707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21090,"internal_metadata":27569566},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":40,"seq":171798691851,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27748,"kb_used_data":784,"kb_used_omap":23,"kb_used_meta":26920,"kb_avail":733975452,"statfs":{"total":751619276800,"available":751590862848,"internally_reserved":0,"allocated":802816,"data_stored":955053,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":24500,"internal_metadata":27566156},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":37,"seq":158913789966,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27696,"kb_used_data":732,"kb_used_omap":23,"kb_used_meta":26920,"kb_avail":733975504,"statfs":{"total":751619276800,"available":751590916096,"internally_reserved":0,"allocated":749568,"data_stored":364081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24460,"internal_metadata":27566196},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":30,"seq":128849018895,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27756,"kb_used_data":792,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":733975444,"statfs":{"total":751619276800,"available":751590854656,"internally_reserved":0,"allocated":811008,"data_stored":954631,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":29636,"internal_metadata":27561020},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":26,"seq":111669149712,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27688,"kb_used_data":724,"kb_used_omap":23,"kb_used_meta":26920,"kb_avail":733975512,"statfs":{"total":751619276800,"available":751590924288,"internally_reserved":0,"allocated":741376,"data_stored":355747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24232,"internal_metadata":27566424},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":19,"seq":81604378642,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27704,"kb_used_data":740,"kb_used_omap":39,"kb_used_meta":26904,"kb_avail":733975496,"statfs":{"total":751619276800,"available":751590907904,"internally_reserved":0,"allocated":757760,"data_stored":364185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":40627,"internal_metadata":27550029},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27712,"kb_used_data":748,"kb_used_omap":48,"kb_used_meta":26895,"kb_avail":733975488,"statfs":{"total":751619276800,"available":751590899712,"internally_reserved":0,"allocated":765952,"data_stored":365749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49734,"internal_metadata":27540922},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672981,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27708,"kb_used_data":744,"kb_used_omap":38,"kb_used_meta":26905,"kb_avail":733975492,"statfs":{"total":751619276800,"available":751590903808,"internally_reserved":0,"allocated":761856,"data_stored":365715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39406,"internal_metadata":27551250},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":7,"seq":30064771094,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27700,"kb_used_data":736,"kb_used_omap":40,"kb_used_meta":26903,"kb_avail":733975500,"statfs":{"total":751619276800,"available":751590912000,"internally_reserved":0,"allocated":753664,"data_stored":364171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41083,"internal_metadata":27549573},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":36563,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":36563,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20097,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":4103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":705,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13826,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":426,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1256,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":366,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1586,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":656,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4234,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":470,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":8579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17757,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3990,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-13T02:22:24.245 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph pg dump --format=json 2026-02-13T02:22:24.300 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:24 trial027 ceph-mon[24567]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:22:24.300 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:24 trial027 ceph-mon[24567]: pgmap v187: 97 pgs: 97 active+clean; 586 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2026-02-13T02:22:24.300 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:24 trial027 ceph-mon[24567]: from='client.14841 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:22:24.300 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:24 trial027 ceph-mon[24567]: mgrmap e12: x(active, since 2m), standbys: z, y 2026-02-13T02:22:24.300 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:24 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/4030777049' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-13T02:22:24.301 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:24 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/4281799962' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-13T02:22:24.393 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:24.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:24 trial028 ceph-mon[26701]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:22:24.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:24 trial028 ceph-mon[26701]: pgmap v187: 97 pgs: 97 active+clean; 586 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2026-02-13T02:22:24.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:24 trial028 ceph-mon[26701]: from='client.14841 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:22:24.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:24 trial028 ceph-mon[26701]: mgrmap e12: x(active, since 2m), standbys: z, y 2026-02-13T02:22:24.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:24 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/4030777049' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-13T02:22:24.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:24 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/4281799962' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-13T02:22:24.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:24 trial107 ceph-mon[27422]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:22:24.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:24 trial107 ceph-mon[27422]: pgmap v187: 97 pgs: 97 active+clean; 586 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2026-02-13T02:22:24.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:24 trial107 ceph-mon[27422]: from='client.14841 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2026-02-13T02:22:24.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:24 trial107 ceph-mon[27422]: mgrmap e12: x(active, since 2m), standbys: z, y 2026-02-13T02:22:24.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:24 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/4030777049' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-13T02:22:24.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:24 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/4281799962' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-13T02:22:24.506 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.503+0000 7fc3d12246c0 1 -- 10.20.193.27:0/2334314612 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc3cc10f230 msgr2=0x7fc3cc10f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:24.506 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.503+0000 7fc3d12246c0 1 --2- 10.20.193.27:0/2334314612 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc3cc10f230 0x7fc3cc10f600 secure :-1 s=READY pgs=247 gs=1 cs=0 l=1 c_cookie=11625691906581913470 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc3c00099a0 tx=0x7fc3c002ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:24.506 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.503+0000 7fc3d12246c0 1 -- 10.20.193.27:0/2334314612 shutdown_connections 2026-02-13T02:22:24.507 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.503+0000 7fc3d12246c0 1 --2- 10.20.193.27:0/2334314612 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc3cc115440 0x7fc3cc175de0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:24.507 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.503+0000 7fc3d12246c0 1 --2- 10.20.193.27:0/2334314612 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc3cc10fc20 0x7fc3cc114c60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:24.507 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.503+0000 7fc3d12246c0 1 --2- 10.20.193.27:0/2334314612 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc3cc10f230 0x7fc3cc10f600 unknown :-1 s=CLOSED pgs=247 gs=1 cs=0 l=1 c_cookie=11625691906581913470 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:24.507 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.503+0000 7fc3d12246c0 1 -- 10.20.193.27:0/2334314612 >> 10.20.193.27:0/2334314612 conn(0x7fc3cc0ecb30 msgr2=0x7fc3cc10cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:24.507 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.503+0000 7fc3d12246c0 1 -- 10.20.193.27:0/2334314612 shutdown_connections 2026-02-13T02:22:24.507 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.503+0000 7fc3d12246c0 1 -- 10.20.193.27:0/2334314612 wait complete. 2026-02-13T02:22:24.507 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.503+0000 7fc3d12246c0 1 Processor -- start 2026-02-13T02:22:24.507 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.503+0000 7fc3d12246c0 1 -- start start 2026-02-13T02:22:24.507 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.503+0000 7fc3d12246c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc3cc10f230 0x7fc3cc11d3b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:24.507 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.503+0000 7fc3d12246c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc3cc115440 0x7fc3cc11d8f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:24.508 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.503+0000 7fc3d12246c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc3cc122cb0 0x7fc3cc123150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:24.508 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.503+0000 7fc3d12246c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc1236c0 con 0x7fc3cc115440 2026-02-13T02:22:24.508 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.503+0000 7fc3d12246c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc123830 con 0x7fc3cc10f230 2026-02-13T02:22:24.508 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.503+0000 7fc3d12246c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc1239f0 con 0x7fc3cc122cb0 2026-02-13T02:22:24.508 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.503+0000 7fc3ca5756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc3cc115440 0x7fc3cc11d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:24.508 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.503+0000 7fc3ca5756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc3cc115440 0x7fc3cc11d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55426/0 (socket says 10.20.193.27:55426) 2026-02-13T02:22:24.508 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.503+0000 7fc3ca5756c0 1 -- 10.20.193.27:0/511139793 learned_addr learned my addr 10.20.193.27:0/511139793 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:24.508 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.503+0000 7fc3cb5776c0 1 --2- 10.20.193.27:0/511139793 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc3cc122cb0 0x7fc3cc123150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:24.508 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.503+0000 7fc3cad766c0 1 --2- 10.20.193.27:0/511139793 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc3cc10f230 0x7fc3cc11d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:24.508 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.503+0000 7fc3ca5756c0 1 -- 10.20.193.27:0/511139793 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc3cc122cb0 msgr2=0x7fc3cc123150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:24.508 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.503+0000 7fc3ca5756c0 1 --2- 10.20.193.27:0/511139793 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc3cc122cb0 0x7fc3cc123150 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:24.508 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.503+0000 7fc3ca5756c0 1 -- 10.20.193.27:0/511139793 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc3cc10f230 msgr2=0x7fc3cc11d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:24.509 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.503+0000 7fc3ca5756c0 1 --2- 10.20.193.27:0/511139793 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc3cc10f230 0x7fc3cc11d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:24.509 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.503+0000 7fc3ca5756c0 1 -- 10.20.193.27:0/511139793 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c0009590 con 0x7fc3cc115440 2026-02-13T02:22:24.509 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.504+0000 7fc3ca5756c0 1 --2- 10.20.193.27:0/511139793 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc3cc115440 0x7fc3cc11d8f0 secure :-1 s=READY pgs=248 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc3b4005dc0 tx=0x7fc3b400bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:24.509 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.504+0000 7fc3abfff6c0 1 -- 10.20.193.27:0/511139793 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc3b4004250 con 0x7fc3cc115440 2026-02-13T02:22:24.509 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.504+0000 7fc3abfff6c0 1 -- 10.20.193.27:0/511139793 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc3b4009070 con 0x7fc3cc115440 2026-02-13T02:22:24.509 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.504+0000 7fc3d12246c0 1 -- 10.20.193.27:0/511139793 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3cc163ea0 con 0x7fc3cc115440 2026-02-13T02:22:24.509 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.504+0000 7fc3abfff6c0 1 -- 10.20.193.27:0/511139793 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc3b400cd50 con 0x7fc3cc115440 2026-02-13T02:22:24.509 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.504+0000 7fc3d12246c0 1 -- 10.20.193.27:0/511139793 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3cc164330 con 0x7fc3cc115440 2026-02-13T02:22:24.509 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.504+0000 7fc3d12246c0 1 -- 10.20.193.27:0/511139793 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3cc110500 con 0x7fc3cc115440 2026-02-13T02:22:24.510 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.506+0000 7fc3abfff6c0 1 -- 10.20.193.27:0/511139793 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7fc3b40028c0 con 0x7fc3cc115440 2026-02-13T02:22:24.510 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.506+0000 7fc3abfff6c0 1 --2- 10.20.193.27:0/511139793 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc3a00b5690 0x7fc3a00d5b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:24.511 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.506+0000 7fc3cad766c0 1 --2- 10.20.193.27:0/511139793 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc3a00b5690 0x7fc3a00d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:24.511 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.507+0000 7fc3abfff6c0 1 -- 10.20.193.27:0/511139793 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7fc3b40e3bf0 con 0x7fc3cc115440 2026-02-13T02:22:24.511 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.507+0000 7fc3cad766c0 1 --2- 10.20.193.27:0/511139793 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc3a00b5690 0x7fc3a00d5b40 secure :-1 s=READY pgs=179 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc3c00092e0 tx=0x7fc3c0005c00 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:24.511 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.507+0000 7fc3abfff6c0 1 -- 10.20.193.27:0/511139793 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fc3b40a6610 con 0x7fc3cc115440 2026-02-13T02:22:24.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.597+0000 7fc3d12246c0 1 -- 10.20.193.27:0/511139793 --> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc3cc11e500 con 0x7fc3a00b5690 2026-02-13T02:22:24.600 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.599+0000 7fc3abfff6c0 1 -- 10.20.193.27:0/511139793 <== mgr.14116 v2:10.20.193.27:6800/1837452956 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258825 (secure 0 0 0) 0x7fc3cc11e500 con 0x7fc3a00b5690 2026-02-13T02:22:24.603 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:22:24.605 INFO:teuthology.orchestra.run.trial027.stderr:dumped all 2026-02-13T02:22:24.606 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.601+0000 7fc3d12246c0 1 -- 10.20.193.27:0/511139793 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc3a00b5690 msgr2=0x7fc3a00d5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:24.607 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.601+0000 7fc3d12246c0 1 --2- 10.20.193.27:0/511139793 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc3a00b5690 0x7fc3a00d5b40 secure :-1 s=READY pgs=179 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc3c00092e0 tx=0x7fc3c0005c00 comp rx=0 tx=0).stop 2026-02-13T02:22:24.607 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.601+0000 7fc3d12246c0 1 -- 10.20.193.27:0/511139793 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc3cc115440 msgr2=0x7fc3cc11d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:24.607 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.601+0000 7fc3d12246c0 1 --2- 10.20.193.27:0/511139793 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc3cc115440 0x7fc3cc11d8f0 secure :-1 s=READY pgs=248 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc3b4005dc0 tx=0x7fc3b400bbb0 comp rx=0 tx=0).stop 2026-02-13T02:22:24.607 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.601+0000 7fc3d12246c0 1 -- 10.20.193.27:0/511139793 shutdown_connections 2026-02-13T02:22:24.607 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.601+0000 7fc3d12246c0 1 --2- 10.20.193.27:0/511139793 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc3a00b5690 0x7fc3a00d5b40 unknown :-1 s=CLOSED pgs=179 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:24.607 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.601+0000 7fc3d12246c0 1 --2- 10.20.193.27:0/511139793 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc3cc122cb0 0x7fc3cc123150 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:24.607 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.601+0000 7fc3d12246c0 1 --2- 10.20.193.27:0/511139793 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc3cc115440 0x7fc3cc11d8f0 unknown :-1 s=CLOSED pgs=248 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:24.607 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.601+0000 7fc3d12246c0 1 --2- 10.20.193.27:0/511139793 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc3cc10f230 0x7fc3cc11d3b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:24.607 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.601+0000 7fc3d12246c0 1 -- 10.20.193.27:0/511139793 >> 10.20.193.27:0/511139793 conn(0x7fc3cc0ecb30 msgr2=0x7fc3cc10cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:24.607 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.601+0000 7fc3d12246c0 1 -- 10.20.193.27:0/511139793 shutdown_connections 2026-02-13T02:22:24.607 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.601+0000 7fc3d12246c0 1 -- 10.20.193.27:0/511139793 wait complete. 2026-02-13T02:22:24.647 INFO:teuthology.orchestra.run.trial027.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2026-02-13T02:22:22.957076+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":599573,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":115,"num_read_kb":98,"num_write":237,"num_write_kb":3739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":194,"ondisk_log_size":194,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":260,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":8808038400,"kb_used":332580,"kb_used_data":9012,"kb_used_omap":327,"kb_used_meta":323000,"kb_avail":8807705820,"statfs":{"total":9019431321600,"available":9019090759680,"internally_reserved":0,"allocated":9228288,"data_stored":6139082,"data_compressed":114621,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":335075,"internal_metadata":330752797},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9205,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":9,"num_read_kb":7,"num_write":48,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.953679"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016636+0000","last_change":"2026-02-13T02:22:17.023288+0000","last_active":"2026-02-13T02:22:18.016636+0000","last_peered":"2026-02-13T02:22:18.016636+0000","last_clean":"2026-02-13T02:22:18.016636+0000","last_became_active":"2026-02-13T02:22:17.023073+0000","last_became_peered":"2026-02-13T02:22:17.023073+0000","last_unstale":"2026-02-13T02:22:18.016636+0000","last_undegraded":"2026-02-13T02:22:18.016636+0000","last_fullsized":"2026-02-13T02:22:18.016636+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:19:00.322079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047477+0000","last_change":"2026-02-13T02:22:17.018633+0000","last_active":"2026-02-13T02:22:18.047477+0000","last_peered":"2026-02-13T02:22:18.047477+0000","last_clean":"2026-02-13T02:22:18.047477+0000","last_became_active":"2026-02-13T02:22:17.018516+0000","last_became_peered":"2026-02-13T02:22:17.018516+0000","last_unstale":"2026-02-13T02:22:18.047477+0000","last_undegraded":"2026-02-13T02:22:18.047477+0000","last_fullsized":"2026-02-13T02:22:18.047477+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:33:08.892207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047500+0000","last_change":"2026-02-13T02:22:17.019231+0000","last_active":"2026-02-13T02:22:18.047500+0000","last_peered":"2026-02-13T02:22:18.047500+0000","last_clean":"2026-02-13T02:22:18.047500+0000","last_became_active":"2026-02-13T02:22:17.019039+0000","last_became_peered":"2026-02-13T02:22:17.019039+0000","last_unstale":"2026-02-13T02:22:18.047500+0000","last_undegraded":"2026-02-13T02:22:18.047500+0000","last_fullsized":"2026-02-13T02:22:18.047500+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:21:31.573485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2026-02-13T02:22:17.022783+0000","last_change":"2026-02-13T02:22:17.021960+0000","last_active":"2026-02-13T02:22:17.022783+0000","last_peered":"2026-02-13T02:22:17.022783+0000","last_clean":"2026-02-13T02:22:17.022783+0000","last_became_active":"2026-02-13T02:22:17.021325+0000","last_became_peered":"2026-02-13T02:22:17.021325+0000","last_unstale":"2026-02-13T02:22:17.022783+0000","last_undegraded":"2026-02-13T02:22:17.022783+0000","last_fullsized":"2026-02-13T02:22:17.022783+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:56:25.616661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.024818+0000","last_change":"2026-02-13T02:22:17.022417+0000","last_active":"2026-02-13T02:22:18.024818+0000","last_peered":"2026-02-13T02:22:18.024818+0000","last_clean":"2026-02-13T02:22:18.024818+0000","last_became_active":"2026-02-13T02:22:17.022150+0000","last_became_peered":"2026-02-13T02:22:17.022150+0000","last_unstale":"2026-02-13T02:22:18.024818+0000","last_undegraded":"2026-02-13T02:22:18.024818+0000","last_fullsized":"2026-02-13T02:22:18.024818+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T12:26:21.518119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051355+0000","last_change":"2026-02-13T02:22:17.024197+0000","last_active":"2026-02-13T02:22:18.051355+0000","last_peered":"2026-02-13T02:22:18.051355+0000","last_clean":"2026-02-13T02:22:18.051355+0000","last_became_active":"2026-02-13T02:22:17.023378+0000","last_became_peered":"2026-02-13T02:22:17.023378+0000","last_unstale":"2026-02-13T02:22:18.051355+0000","last_undegraded":"2026-02-13T02:22:18.051355+0000","last_fullsized":"2026-02-13T02:22:18.051355+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:17:40.942731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051199+0000","last_change":"2026-02-13T02:22:17.021184+0000","last_active":"2026-02-13T02:22:18.051199+0000","last_peered":"2026-02-13T02:22:18.051199+0000","last_clean":"2026-02-13T02:22:18.051199+0000","last_became_active":"2026-02-13T02:22:17.020961+0000","last_became_peered":"2026-02-13T02:22:17.020961+0000","last_unstale":"2026-02-13T02:22:18.051199+0000","last_undegraded":"2026-02-13T02:22:18.051199+0000","last_fullsized":"2026-02-13T02:22:18.051199+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T10:38:19.326702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.020937+0000","last_change":"2026-02-13T02:22:17.021948+0000","last_active":"2026-02-13T02:22:18.020937+0000","last_peered":"2026-02-13T02:22:18.020937+0000","last_clean":"2026-02-13T02:22:18.020937+0000","last_became_active":"2026-02-13T02:22:17.021760+0000","last_became_peered":"2026-02-13T02:22:17.021760+0000","last_unstale":"2026-02-13T02:22:18.020937+0000","last_undegraded":"2026-02-13T02:22:18.020937+0000","last_fullsized":"2026-02-13T02:22:18.020937+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:24:58.456659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051727+0000","last_change":"2026-02-13T02:22:17.024036+0000","last_active":"2026-02-13T02:22:18.051727+0000","last_peered":"2026-02-13T02:22:18.051727+0000","last_clean":"2026-02-13T02:22:18.051727+0000","last_became_active":"2026-02-13T02:22:17.023805+0000","last_became_peered":"2026-02-13T02:22:17.023805+0000","last_unstale":"2026-02-13T02:22:18.051727+0000","last_undegraded":"2026-02-13T02:22:18.051727+0000","last_fullsized":"2026-02-13T02:22:18.051727+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T02:44:29.738084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051866+0000","last_change":"2026-02-13T02:22:17.024557+0000","last_active":"2026-02-13T02:22:18.051866+0000","last_peered":"2026-02-13T02:22:18.051866+0000","last_clean":"2026-02-13T02:22:18.051866+0000","last_became_active":"2026-02-13T02:22:17.024270+0000","last_became_peered":"2026-02-13T02:22:17.024270+0000","last_unstale":"2026-02-13T02:22:18.051866+0000","last_undegraded":"2026-02-13T02:22:18.051866+0000","last_fullsized":"2026-02-13T02:22:18.051866+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:36:05.815116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051729+0000","last_change":"2026-02-13T02:22:17.023942+0000","last_active":"2026-02-13T02:22:18.051729+0000","last_peered":"2026-02-13T02:22:18.051729+0000","last_clean":"2026-02-13T02:22:18.051729+0000","last_became_active":"2026-02-13T02:22:17.023475+0000","last_became_peered":"2026-02-13T02:22:17.023475+0000","last_unstale":"2026-02-13T02:22:18.051729+0000","last_undegraded":"2026-02-13T02:22:18.051729+0000","last_fullsized":"2026-02-13T02:22:18.051729+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T12:17:44.718186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":69,"state":"active+clean","last_fresh":"2026-02-13T02:22:17.023199+0000","last_change":"2026-02-13T02:22:17.022471+0000","last_active":"2026-02-13T02:22:17.023199+0000","last_peered":"2026-02-13T02:22:17.023199+0000","last_clean":"2026-02-13T02:22:17.023199+0000","last_became_active":"2026-02-13T02:22:17.021894+0000","last_became_peered":"2026-02-13T02:22:17.021894+0000","last_unstale":"2026-02-13T02:22:17.023199+0000","last_undegraded":"2026-02-13T02:22:17.023199+0000","last_fullsized":"2026-02-13T02:22:17.023199+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T10:41:27.319466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047429+0000","last_change":"2026-02-13T02:22:17.017830+0000","last_active":"2026-02-13T02:22:18.047429+0000","last_peered":"2026-02-13T02:22:18.047429+0000","last_clean":"2026-02-13T02:22:18.047429+0000","last_became_active":"2026-02-13T02:22:17.017232+0000","last_became_peered":"2026-02-13T02:22:17.017232+0000","last_unstale":"2026-02-13T02:22:18.047429+0000","last_undegraded":"2026-02-13T02:22:18.047429+0000","last_fullsized":"2026-02-13T02:22:18.047429+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:40:47.725661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047355+0000","last_change":"2026-02-13T02:22:17.021764+0000","last_active":"2026-02-13T02:22:18.047355+0000","last_peered":"2026-02-13T02:22:18.047355+0000","last_clean":"2026-02-13T02:22:18.047355+0000","last_became_active":"2026-02-13T02:22:17.021609+0000","last_became_peered":"2026-02-13T02:22:17.021609+0000","last_unstale":"2026-02-13T02:22:18.047355+0000","last_undegraded":"2026-02-13T02:22:18.047355+0000","last_fullsized":"2026-02-13T02:22:18.047355+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T12:04:31.168034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047332+0000","last_change":"2026-02-13T02:22:17.019584+0000","last_active":"2026-02-13T02:22:18.047332+0000","last_peered":"2026-02-13T02:22:18.047332+0000","last_clean":"2026-02-13T02:22:18.047332+0000","last_became_active":"2026-02-13T02:22:17.019388+0000","last_became_peered":"2026-02-13T02:22:17.019388+0000","last_unstale":"2026-02-13T02:22:18.047332+0000","last_undegraded":"2026-02-13T02:22:18.047332+0000","last_fullsized":"2026-02-13T02:22:18.047332+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:33:16.084048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.050998+0000","last_change":"2026-02-13T02:22:17.023950+0000","last_active":"2026-02-13T02:22:18.050998+0000","last_peered":"2026-02-13T02:22:18.050998+0000","last_clean":"2026-02-13T02:22:18.050998+0000","last_became_active":"2026-02-13T02:22:17.023029+0000","last_became_peered":"2026-02-13T02:22:17.023029+0000","last_unstale":"2026-02-13T02:22:18.050998+0000","last_undegraded":"2026-02-13T02:22:18.050998+0000","last_fullsized":"2026-02-13T02:22:18.050998+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:14:25.119381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016670+0000","last_change":"2026-02-13T02:22:17.024675+0000","last_active":"2026-02-13T02:22:18.016670+0000","last_peered":"2026-02-13T02:22:18.016670+0000","last_clean":"2026-02-13T02:22:18.016670+0000","last_became_active":"2026-02-13T02:22:17.024460+0000","last_became_peered":"2026-02-13T02:22:17.024460+0000","last_unstale":"2026-02-13T02:22:18.016670+0000","last_undegraded":"2026-02-13T02:22:18.016670+0000","last_fullsized":"2026-02-13T02:22:18.016670+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T04:41:20.995149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.012018+0000","last_change":"2026-02-13T02:22:17.019261+0000","last_active":"2026-02-13T02:22:18.012018+0000","last_peered":"2026-02-13T02:22:18.012018+0000","last_clean":"2026-02-13T02:22:18.012018+0000","last_became_active":"2026-02-13T02:22:17.019084+0000","last_became_peered":"2026-02-13T02:22:17.019084+0000","last_unstale":"2026-02-13T02:22:18.012018+0000","last_undegraded":"2026-02-13T02:22:18.012018+0000","last_fullsized":"2026-02-13T02:22:18.012018+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:26:06.852915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016186+0000","last_change":"2026-02-13T02:22:17.021336+0000","last_active":"2026-02-13T02:22:18.016186+0000","last_peered":"2026-02-13T02:22:18.016186+0000","last_clean":"2026-02-13T02:22:18.016186+0000","last_became_active":"2026-02-13T02:22:17.020885+0000","last_became_peered":"2026-02-13T02:22:17.020885+0000","last_unstale":"2026-02-13T02:22:18.016186+0000","last_undegraded":"2026-02-13T02:22:18.016186+0000","last_fullsized":"2026-02-13T02:22:18.016186+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T14:05:29.982317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016382+0000","last_change":"2026-02-13T02:22:17.022900+0000","last_active":"2026-02-13T02:22:18.016382+0000","last_peered":"2026-02-13T02:22:18.016382+0000","last_clean":"2026-02-13T02:22:18.016382+0000","last_became_active":"2026-02-13T02:22:17.022753+0000","last_became_peered":"2026-02-13T02:22:17.022753+0000","last_unstale":"2026-02-13T02:22:18.016382+0000","last_undegraded":"2026-02-13T02:22:18.016382+0000","last_fullsized":"2026-02-13T02:22:18.016382+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:35:07.485606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047442+0000","last_change":"2026-02-13T02:22:17.018325+0000","last_active":"2026-02-13T02:22:18.047442+0000","last_peered":"2026-02-13T02:22:18.047442+0000","last_clean":"2026-02-13T02:22:18.047442+0000","last_became_active":"2026-02-13T02:22:17.018167+0000","last_became_peered":"2026-02-13T02:22:17.018167+0000","last_unstale":"2026-02-13T02:22:18.047442+0000","last_undegraded":"2026-02-13T02:22:18.047442+0000","last_fullsized":"2026-02-13T02:22:18.047442+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T14:09:03.876188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051256+0000","last_change":"2026-02-13T02:22:17.022546+0000","last_active":"2026-02-13T02:22:18.051256+0000","last_peered":"2026-02-13T02:22:18.051256+0000","last_clean":"2026-02-13T02:22:18.051256+0000","last_became_active":"2026-02-13T02:22:17.022064+0000","last_became_peered":"2026-02-13T02:22:17.022064+0000","last_unstale":"2026-02-13T02:22:18.051256+0000","last_undegraded":"2026-02-13T02:22:18.051256+0000","last_fullsized":"2026-02-13T02:22:18.051256+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:55:49.327407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.020899+0000","last_change":"2026-02-13T02:22:17.018592+0000","last_active":"2026-02-13T02:22:18.020899+0000","last_peered":"2026-02-13T02:22:18.020899+0000","last_clean":"2026-02-13T02:22:18.020899+0000","last_became_active":"2026-02-13T02:22:17.018368+0000","last_became_peered":"2026-02-13T02:22:17.018368+0000","last_unstale":"2026-02-13T02:22:18.020899+0000","last_undegraded":"2026-02-13T02:22:18.020899+0000","last_fullsized":"2026-02-13T02:22:18.020899+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:15:11.963709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051251+0000","last_change":"2026-02-13T02:22:17.023462+0000","last_active":"2026-02-13T02:22:18.051251+0000","last_peered":"2026-02-13T02:22:18.051251+0000","last_clean":"2026-02-13T02:22:18.051251+0000","last_became_active":"2026-02-13T02:22:17.023327+0000","last_became_peered":"2026-02-13T02:22:17.023327+0000","last_unstale":"2026-02-13T02:22:18.051251+0000","last_undegraded":"2026-02-13T02:22:18.051251+0000","last_fullsized":"2026-02-13T02:22:18.051251+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:42:27.909493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.021321+0000","last_change":"2026-02-13T02:22:17.021211+0000","last_active":"2026-02-13T02:22:18.021321+0000","last_peered":"2026-02-13T02:22:18.021321+0000","last_clean":"2026-02-13T02:22:18.021321+0000","last_became_active":"2026-02-13T02:22:17.021002+0000","last_became_peered":"2026-02-13T02:22:17.021002+0000","last_unstale":"2026-02-13T02:22:18.021321+0000","last_undegraded":"2026-02-13T02:22:18.021321+0000","last_fullsized":"2026-02-13T02:22:18.021321+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:23:00.242280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016321+0000","last_change":"2026-02-13T02:22:17.023595+0000","last_active":"2026-02-13T02:22:18.016321+0000","last_peered":"2026-02-13T02:22:18.016321+0000","last_clean":"2026-02-13T02:22:18.016321+0000","last_became_active":"2026-02-13T02:22:17.023480+0000","last_became_peered":"2026-02-13T02:22:17.023480+0000","last_unstale":"2026-02-13T02:22:18.016321+0000","last_undegraded":"2026-02-13T02:22:18.016321+0000","last_fullsized":"2026-02-13T02:22:18.016321+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:13:14.505114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016555+0000","last_change":"2026-02-13T02:22:17.021599+0000","last_active":"2026-02-13T02:22:18.016555+0000","last_peered":"2026-02-13T02:22:18.016555+0000","last_clean":"2026-02-13T02:22:18.016555+0000","last_became_active":"2026-02-13T02:22:17.021373+0000","last_became_peered":"2026-02-13T02:22:17.021373+0000","last_unstale":"2026-02-13T02:22:18.016555+0000","last_undegraded":"2026-02-13T02:22:18.016555+0000","last_fullsized":"2026-02-13T02:22:18.016555+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:35:13.390476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047383+0000","last_change":"2026-02-13T02:22:17.017507+0000","last_active":"2026-02-13T02:22:18.047383+0000","last_peered":"2026-02-13T02:22:18.047383+0000","last_clean":"2026-02-13T02:22:18.047383+0000","last_became_active":"2026-02-13T02:22:17.017211+0000","last_became_peered":"2026-02-13T02:22:17.017211+0000","last_unstale":"2026-02-13T02:22:18.047383+0000","last_undegraded":"2026-02-13T02:22:18.047383+0000","last_fullsized":"2026-02-13T02:22:18.047383+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:19:19.531222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047561+0000","last_change":"2026-02-13T02:22:17.019486+0000","last_active":"2026-02-13T02:22:18.047561+0000","last_peered":"2026-02-13T02:22:18.047561+0000","last_clean":"2026-02-13T02:22:18.047561+0000","last_became_active":"2026-02-13T02:22:17.019288+0000","last_became_peered":"2026-02-13T02:22:17.019288+0000","last_unstale":"2026-02-13T02:22:18.047561+0000","last_undegraded":"2026-02-13T02:22:18.047561+0000","last_fullsized":"2026-02-13T02:22:18.047561+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:51:14.055654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.015982+0000","last_change":"2026-02-13T02:22:17.022029+0000","last_active":"2026-02-13T02:22:18.015982+0000","last_peered":"2026-02-13T02:22:18.015982+0000","last_clean":"2026-02-13T02:22:18.015982+0000","last_became_active":"2026-02-13T02:22:17.021771+0000","last_became_peered":"2026-02-13T02:22:17.021771+0000","last_unstale":"2026-02-13T02:22:18.015982+0000","last_undegraded":"2026-02-13T02:22:18.015982+0000","last_fullsized":"2026-02-13T02:22:18.015982+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:15:51.936117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016131+0000","last_change":"2026-02-13T02:22:17.021554+0000","last_active":"2026-02-13T02:22:18.016131+0000","last_peered":"2026-02-13T02:22:18.016131+0000","last_clean":"2026-02-13T02:22:18.016131+0000","last_became_active":"2026-02-13T02:22:17.021248+0000","last_became_peered":"2026-02-13T02:22:17.021248+0000","last_unstale":"2026-02-13T02:22:18.016131+0000","last_undegraded":"2026-02-13T02:22:18.016131+0000","last_fullsized":"2026-02-13T02:22:18.016131+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T12:27:26.132715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016224+0000","last_change":"2026-02-13T02:22:17.022426+0000","last_active":"2026-02-13T02:22:18.016224+0000","last_peered":"2026-02-13T02:22:18.016224+0000","last_clean":"2026-02-13T02:22:18.016224+0000","last_became_active":"2026-02-13T02:22:17.022285+0000","last_became_peered":"2026-02-13T02:22:17.022285+0000","last_unstale":"2026-02-13T02:22:18.016224+0000","last_undegraded":"2026-02-13T02:22:18.016224+0000","last_fullsized":"2026-02-13T02:22:18.016224+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:25:58.151252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.024814+0000","last_change":"2026-02-13T02:22:17.022749+0000","last_active":"2026-02-13T02:22:18.024814+0000","last_peered":"2026-02-13T02:22:18.024814+0000","last_clean":"2026-02-13T02:22:18.024814+0000","last_became_active":"2026-02-13T02:22:17.022397+0000","last_became_peered":"2026-02-13T02:22:17.022397+0000","last_unstale":"2026-02-13T02:22:18.024814+0000","last_undegraded":"2026-02-13T02:22:18.024814+0000","last_fullsized":"2026-02-13T02:22:18.024814+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:28:44.768608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"69'1","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051275+0000","last_change":"2026-02-13T02:22:16.014721+0000","last_active":"2026-02-13T02:22:18.051275+0000","last_peered":"2026-02-13T02:22:18.051275+0000","last_clean":"2026-02-13T02:22:18.051275+0000","last_became_active":"2026-02-13T02:22:16.014587+0000","last_became_peered":"2026-02-13T02:22:16.014587+0000","last_unstale":"2026-02-13T02:22:18.051275+0000","last_undegraded":"2026-02-13T02:22:18.051275+0000","last_fullsized":"2026-02-13T02:22:18.051275+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:56:58.003775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051261+0000","last_change":"2026-02-13T02:22:17.022514+0000","last_active":"2026-02-13T02:22:18.051261+0000","last_peered":"2026-02-13T02:22:18.051261+0000","last_clean":"2026-02-13T02:22:18.051261+0000","last_became_active":"2026-02-13T02:22:17.022273+0000","last_became_peered":"2026-02-13T02:22:17.022273+0000","last_unstale":"2026-02-13T02:22:18.051261+0000","last_undegraded":"2026-02-13T02:22:18.051261+0000","last_fullsized":"2026-02-13T02:22:18.051261+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:33:27.805740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"69'1","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051311+0000","last_change":"2026-02-13T02:22:16.012690+0000","last_active":"2026-02-13T02:22:18.051311+0000","last_peered":"2026-02-13T02:22:18.051311+0000","last_clean":"2026-02-13T02:22:18.051311+0000","last_became_active":"2026-02-13T02:22:16.011935+0000","last_became_peered":"2026-02-13T02:22:16.011935+0000","last_unstale":"2026-02-13T02:22:18.051311+0000","last_undegraded":"2026-02-13T02:22:18.051311+0000","last_fullsized":"2026-02-13T02:22:18.051311+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T06:55:46.440839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047121+0000","last_change":"2026-02-13T02:22:16.007738+0000","last_active":"2026-02-13T02:22:18.047121+0000","last_peered":"2026-02-13T02:22:18.047121+0000","last_clean":"2026-02-13T02:22:18.047121+0000","last_became_active":"2026-02-13T02:22:16.007424+0000","last_became_peered":"2026-02-13T02:22:16.007424+0000","last_unstale":"2026-02-13T02:22:18.047121+0000","last_undegraded":"2026-02-13T02:22:18.047121+0000","last_fullsized":"2026-02-13T02:22:18.047121+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:10:41.888980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.020958+0000","last_change":"2026-02-13T02:22:17.017332+0000","last_active":"2026-02-13T02:22:18.020958+0000","last_peered":"2026-02-13T02:22:18.020958+0000","last_clean":"2026-02-13T02:22:18.020958+0000","last_became_active":"2026-02-13T02:22:17.017080+0000","last_became_peered":"2026-02-13T02:22:17.017080+0000","last_unstale":"2026-02-13T02:22:18.020958+0000","last_undegraded":"2026-02-13T02:22:18.020958+0000","last_fullsized":"2026-02-13T02:22:18.020958+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:10:16.349029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051313+0000","last_change":"2026-02-13T02:22:16.014204+0000","last_active":"2026-02-13T02:22:18.051313+0000","last_peered":"2026-02-13T02:22:18.051313+0000","last_clean":"2026-02-13T02:22:18.051313+0000","last_became_active":"2026-02-13T02:22:16.014114+0000","last_became_peered":"2026-02-13T02:22:16.014114+0000","last_unstale":"2026-02-13T02:22:18.051313+0000","last_undegraded":"2026-02-13T02:22:18.051313+0000","last_fullsized":"2026-02-13T02:22:18.051313+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:39:42.557823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.011965+0000","last_change":"2026-02-13T02:22:17.018644+0000","last_active":"2026-02-13T02:22:18.011965+0000","last_peered":"2026-02-13T02:22:18.011965+0000","last_clean":"2026-02-13T02:22:18.011965+0000","last_became_active":"2026-02-13T02:22:17.018511+0000","last_became_peered":"2026-02-13T02:22:17.018511+0000","last_unstale":"2026-02-13T02:22:18.011965+0000","last_undegraded":"2026-02-13T02:22:18.011965+0000","last_fullsized":"2026-02-13T02:22:18.011965+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:18:39.510344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051718+0000","last_change":"2026-02-13T02:22:16.013123+0000","last_active":"2026-02-13T02:22:18.051718+0000","last_peered":"2026-02-13T02:22:18.051718+0000","last_clean":"2026-02-13T02:22:18.051718+0000","last_became_active":"2026-02-13T02:22:16.012986+0000","last_became_peered":"2026-02-13T02:22:16.012986+0000","last_unstale":"2026-02-13T02:22:18.051718+0000","last_undegraded":"2026-02-13T02:22:18.051718+0000","last_fullsized":"2026-02-13T02:22:18.051718+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T10:09:29.447645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051655+0000","last_change":"2026-02-13T02:22:17.024718+0000","last_active":"2026-02-13T02:22:18.051655+0000","last_peered":"2026-02-13T02:22:18.051655+0000","last_clean":"2026-02-13T02:22:18.051655+0000","last_became_active":"2026-02-13T02:22:17.024333+0000","last_became_peered":"2026-02-13T02:22:17.024333+0000","last_unstale":"2026-02-13T02:22:18.051655+0000","last_undegraded":"2026-02-13T02:22:18.051655+0000","last_fullsized":"2026-02-13T02:22:18.051655+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:44:11.055118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051693+0000","last_change":"2026-02-13T02:22:16.015080+0000","last_active":"2026-02-13T02:22:18.051693+0000","last_peered":"2026-02-13T02:22:18.051693+0000","last_clean":"2026-02-13T02:22:18.051693+0000","last_became_active":"2026-02-13T02:22:16.014694+0000","last_became_peered":"2026-02-13T02:22:16.014694+0000","last_unstale":"2026-02-13T02:22:18.051693+0000","last_undegraded":"2026-02-13T02:22:18.051693+0000","last_fullsized":"2026-02-13T02:22:18.051693+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T06:02:28.163616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047603+0000","last_change":"2026-02-13T02:22:17.019516+0000","last_active":"2026-02-13T02:22:18.047603+0000","last_peered":"2026-02-13T02:22:18.047603+0000","last_clean":"2026-02-13T02:22:18.047603+0000","last_became_active":"2026-02-13T02:22:17.019234+0000","last_became_peered":"2026-02-13T02:22:17.019234+0000","last_unstale":"2026-02-13T02:22:18.047603+0000","last_undegraded":"2026-02-13T02:22:18.047603+0000","last_fullsized":"2026-02-13T02:22:18.047603+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T10:25:31.467398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"69'3","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051378+0000","last_change":"2026-02-13T02:22:16.013733+0000","last_active":"2026-02-13T02:22:18.051378+0000","last_peered":"2026-02-13T02:22:18.051378+0000","last_clean":"2026-02-13T02:22:18.051378+0000","last_became_active":"2026-02-13T02:22:16.013472+0000","last_became_peered":"2026-02-13T02:22:16.013472+0000","last_unstale":"2026-02-13T02:22:18.051378+0000","last_undegraded":"2026-02-13T02:22:18.051378+0000","last_fullsized":"2026-02-13T02:22:18.051378+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:59:36.928070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051362+0000","last_change":"2026-02-13T02:22:17.023907+0000","last_active":"2026-02-13T02:22:18.051362+0000","last_peered":"2026-02-13T02:22:18.051362+0000","last_clean":"2026-02-13T02:22:18.051362+0000","last_became_active":"2026-02-13T02:22:17.022420+0000","last_became_peered":"2026-02-13T02:22:17.022420+0000","last_unstale":"2026-02-13T02:22:18.051362+0000","last_undegraded":"2026-02-13T02:22:18.051362+0000","last_fullsized":"2026-02-13T02:22:18.051362+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:29:25.440291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"70'9","reported_seq":31,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:21.436149+0000","last_change":"2026-02-13T02:22:16.013888+0000","last_active":"2026-02-13T02:22:21.436149+0000","last_peered":"2026-02-13T02:22:21.436149+0000","last_clean":"2026-02-13T02:22:21.436149+0000","last_became_active":"2026-02-13T02:22:16.013652+0000","last_became_peered":"2026-02-13T02:22:16.013652+0000","last_unstale":"2026-02-13T02:22:21.436149+0000","last_undegraded":"2026-02-13T02:22:21.436149+0000","last_fullsized":"2026-02-13T02:22:21.436149+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:25:28.136946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8385,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":9,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.025043+0000","last_change":"2026-02-13T02:22:17.023547+0000","last_active":"2026-02-13T02:22:18.025043+0000","last_peered":"2026-02-13T02:22:18.025043+0000","last_clean":"2026-02-13T02:22:18.025043+0000","last_became_active":"2026-02-13T02:22:17.023410+0000","last_became_peered":"2026-02-13T02:22:17.023410+0000","last_unstale":"2026-02-13T02:22:18.025043+0000","last_undegraded":"2026-02-13T02:22:18.025043+0000","last_fullsized":"2026-02-13T02:22:18.025043+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T04:35:03.522242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"69'2","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.015919+0000","last_change":"2026-02-13T02:22:16.014701+0000","last_active":"2026-02-13T02:22:18.015919+0000","last_peered":"2026-02-13T02:22:18.015919+0000","last_clean":"2026-02-13T02:22:18.015919+0000","last_became_active":"2026-02-13T02:22:16.014476+0000","last_became_peered":"2026-02-13T02:22:16.014476+0000","last_unstale":"2026-02-13T02:22:18.015919+0000","last_undegraded":"2026-02-13T02:22:18.015919+0000","last_fullsized":"2026-02-13T02:22:18.015919+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:25:47.865783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047309+0000","last_change":"2026-02-13T02:22:17.017924+0000","last_active":"2026-02-13T02:22:18.047309+0000","last_peered":"2026-02-13T02:22:18.047309+0000","last_clean":"2026-02-13T02:22:18.047309+0000","last_became_active":"2026-02-13T02:22:17.017666+0000","last_became_peered":"2026-02-13T02:22:17.017666+0000","last_unstale":"2026-02-13T02:22:18.047309+0000","last_undegraded":"2026-02-13T02:22:18.047309+0000","last_fullsized":"2026-02-13T02:22:18.047309+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:38:38.063114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051219+0000","last_change":"2026-02-13T02:22:16.014437+0000","last_active":"2026-02-13T02:22:18.051219+0000","last_peered":"2026-02-13T02:22:18.051219+0000","last_clean":"2026-02-13T02:22:18.051219+0000","last_became_active":"2026-02-13T02:22:16.014293+0000","last_became_peered":"2026-02-13T02:22:16.014293+0000","last_unstale":"2026-02-13T02:22:18.051219+0000","last_undegraded":"2026-02-13T02:22:18.051219+0000","last_fullsized":"2026-02-13T02:22:18.051219+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:08:58.340775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051558+0000","last_change":"2026-02-13T02:22:17.023621+0000","last_active":"2026-02-13T02:22:18.051558+0000","last_peered":"2026-02-13T02:22:18.051558+0000","last_clean":"2026-02-13T02:22:18.051558+0000","last_became_active":"2026-02-13T02:22:17.023343+0000","last_became_peered":"2026-02-13T02:22:17.023343+0000","last_unstale":"2026-02-13T02:22:18.051558+0000","last_undegraded":"2026-02-13T02:22:18.051558+0000","last_fullsized":"2026-02-13T02:22:18.051558+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T06:36:08.489377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2026-02-13T02:22:17.015247+0000","last_change":"2026-02-13T02:22:16.013825+0000","last_active":"2026-02-13T02:22:17.015247+0000","last_peered":"2026-02-13T02:22:17.015247+0000","last_clean":"2026-02-13T02:22:17.015247+0000","last_became_active":"2026-02-13T02:22:16.013704+0000","last_became_peered":"2026-02-13T02:22:16.013704+0000","last_unstale":"2026-02-13T02:22:17.015247+0000","last_undegraded":"2026-02-13T02:22:17.015247+0000","last_fullsized":"2026-02-13T02:22:17.015247+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T02:40:53.106423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051148+0000","last_change":"2026-02-13T02:22:17.022293+0000","last_active":"2026-02-13T02:22:18.051148+0000","last_peered":"2026-02-13T02:22:18.051148+0000","last_clean":"2026-02-13T02:22:18.051148+0000","last_became_active":"2026-02-13T02:22:17.022130+0000","last_became_peered":"2026-02-13T02:22:17.022130+0000","last_unstale":"2026-02-13T02:22:18.051148+0000","last_undegraded":"2026-02-13T02:22:18.051148+0000","last_fullsized":"2026-02-13T02:22:18.051148+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:33:03.123178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"69'8","reported_seq":24,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.024641+0000","last_change":"2026-02-13T02:22:16.013612+0000","last_active":"2026-02-13T02:22:18.024641+0000","last_peered":"2026-02-13T02:22:18.024641+0000","last_clean":"2026-02-13T02:22:18.024641+0000","last_became_active":"2026-02-13T02:22:16.013444+0000","last_became_peered":"2026-02-13T02:22:16.013444+0000","last_unstale":"2026-02-13T02:22:18.024641+0000","last_undegraded":"2026-02-13T02:22:18.024641+0000","last_fullsized":"2026-02-13T02:22:18.024641+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T04:59:32.263657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051261+0000","last_change":"2026-02-13T02:22:17.024692+0000","last_active":"2026-02-13T02:22:18.051261+0000","last_peered":"2026-02-13T02:22:18.051261+0000","last_clean":"2026-02-13T02:22:18.051261+0000","last_became_active":"2026-02-13T02:22:17.024543+0000","last_became_peered":"2026-02-13T02:22:17.024543+0000","last_unstale":"2026-02-13T02:22:18.051261+0000","last_undegraded":"2026-02-13T02:22:18.051261+0000","last_fullsized":"2026-02-13T02:22:18.051261+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:04:25.726793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"69'1","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.025073+0000","last_change":"2026-02-13T02:22:16.011678+0000","last_active":"2026-02-13T02:22:18.025073+0000","last_peered":"2026-02-13T02:22:18.025073+0000","last_clean":"2026-02-13T02:22:18.025073+0000","last_became_active":"2026-02-13T02:22:16.010914+0000","last_became_peered":"2026-02-13T02:22:16.010914+0000","last_unstale":"2026-02-13T02:22:18.025073+0000","last_undegraded":"2026-02-13T02:22:18.025073+0000","last_fullsized":"2026-02-13T02:22:18.025073+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:44:29.583923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016496+0000","last_change":"2026-02-13T02:22:17.023312+0000","last_active":"2026-02-13T02:22:18.016496+0000","last_peered":"2026-02-13T02:22:18.016496+0000","last_clean":"2026-02-13T02:22:18.016496+0000","last_became_active":"2026-02-13T02:22:17.023084+0000","last_became_peered":"2026-02-13T02:22:17.023084+0000","last_unstale":"2026-02-13T02:22:18.016496+0000","last_undegraded":"2026-02-13T02:22:18.016496+0000","last_fullsized":"2026-02-13T02:22:18.016496+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T10:27:31.848680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"69'2","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047389+0000","last_change":"2026-02-13T02:22:16.010281+0000","last_active":"2026-02-13T02:22:18.047389+0000","last_peered":"2026-02-13T02:22:18.047389+0000","last_clean":"2026-02-13T02:22:18.047389+0000","last_became_active":"2026-02-13T02:22:16.010075+0000","last_became_peered":"2026-02-13T02:22:16.010075+0000","last_unstale":"2026-02-13T02:22:18.047389+0000","last_undegraded":"2026-02-13T02:22:18.047389+0000","last_fullsized":"2026-02-13T02:22:18.047389+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:25:52.223102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051820+0000","last_change":"2026-02-13T02:22:17.023958+0000","last_active":"2026-02-13T02:22:18.051820+0000","last_peered":"2026-02-13T02:22:18.051820+0000","last_clean":"2026-02-13T02:22:18.051820+0000","last_became_active":"2026-02-13T02:22:17.023529+0000","last_became_peered":"2026-02-13T02:22:17.023529+0000","last_unstale":"2026-02-13T02:22:18.051820+0000","last_undegraded":"2026-02-13T02:22:18.051820+0000","last_fullsized":"2026-02-13T02:22:18.051820+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T12:45:26.937375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"69'1","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051446+0000","last_change":"2026-02-13T02:22:16.013663+0000","last_active":"2026-02-13T02:22:18.051446+0000","last_peered":"2026-02-13T02:22:18.051446+0000","last_clean":"2026-02-13T02:22:18.051446+0000","last_became_active":"2026-02-13T02:22:16.013445+0000","last_became_peered":"2026-02-13T02:22:16.013445+0000","last_unstale":"2026-02-13T02:22:18.051446+0000","last_undegraded":"2026-02-13T02:22:18.051446+0000","last_fullsized":"2026-02-13T02:22:18.051446+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:02:58.149380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051679+0000","last_change":"2026-02-13T02:22:17.021199+0000","last_active":"2026-02-13T02:22:18.051679+0000","last_peered":"2026-02-13T02:22:18.051679+0000","last_clean":"2026-02-13T02:22:18.051679+0000","last_became_active":"2026-02-13T02:22:17.021008+0000","last_became_peered":"2026-02-13T02:22:17.021008+0000","last_unstale":"2026-02-13T02:22:18.051679+0000","last_undegraded":"2026-02-13T02:22:18.051679+0000","last_fullsized":"2026-02-13T02:22:18.051679+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:21:08.010400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016275+0000","last_change":"2026-02-13T02:22:16.017721+0000","last_active":"2026-02-13T02:22:18.016275+0000","last_peered":"2026-02-13T02:22:18.016275+0000","last_clean":"2026-02-13T02:22:18.016275+0000","last_became_active":"2026-02-13T02:22:16.017552+0000","last_became_peered":"2026-02-13T02:22:16.017552+0000","last_unstale":"2026-02-13T02:22:18.016275+0000","last_undegraded":"2026-02-13T02:22:18.016275+0000","last_fullsized":"2026-02-13T02:22:18.016275+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:14:18.034709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051822+0000","last_change":"2026-02-13T02:22:17.025362+0000","last_active":"2026-02-13T02:22:18.051822+0000","last_peered":"2026-02-13T02:22:18.051822+0000","last_clean":"2026-02-13T02:22:18.051822+0000","last_became_active":"2026-02-13T02:22:17.024942+0000","last_became_peered":"2026-02-13T02:22:17.024942+0000","last_unstale":"2026-02-13T02:22:18.051822+0000","last_undegraded":"2026-02-13T02:22:18.051822+0000","last_fullsized":"2026-02-13T02:22:18.051822+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:09:37.454791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'152","reported_seq":339,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051251+0000","last_change":"2026-02-13T02:21:55.384973+0000","last_active":"2026-02-13T02:22:18.051251+0000","last_peered":"2026-02-13T02:22:18.051251+0000","last_clean":"2026-02-13T02:22:18.051251+0000","last_became_active":"2026-02-13T02:21:55.384610+0000","last_became_peered":"2026-02-13T02:21:55.384610+0000","last_unstale":"2026-02-13T02:22:18.051251+0000","last_undegraded":"2026-02-13T02:22:18.051251+0000","last_fullsized":"2026-02-13T02:22:18.051251+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:20:55.621197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:20:55.621197+0000","last_clean_scrub_stamp":"2026-02-13T02:20:55.621197+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:45:14.548172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":189,"num_write_kb":3716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2026-02-13T02:22:17.015114+0000","last_change":"2026-02-13T02:22:16.012172+0000","last_active":"2026-02-13T02:22:17.015114+0000","last_peered":"2026-02-13T02:22:17.015114+0000","last_clean":"2026-02-13T02:22:17.015114+0000","last_became_active":"2026-02-13T02:22:16.011646+0000","last_became_peered":"2026-02-13T02:22:16.011646+0000","last_unstale":"2026-02-13T02:22:17.015114+0000","last_undegraded":"2026-02-13T02:22:17.015114+0000","last_fullsized":"2026-02-13T02:22:17.015114+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:05:01.007415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.021079+0000","last_change":"2026-02-13T02:22:17.020741+0000","last_active":"2026-02-13T02:22:18.021079+0000","last_peered":"2026-02-13T02:22:18.021079+0000","last_clean":"2026-02-13T02:22:18.021079+0000","last_became_active":"2026-02-13T02:22:17.020574+0000","last_became_peered":"2026-02-13T02:22:17.020574+0000","last_unstale":"2026-02-13T02:22:18.021079+0000","last_undegraded":"2026-02-13T02:22:18.021079+0000","last_fullsized":"2026-02-13T02:22:18.021079+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T06:28:11.279493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051246+0000","last_change":"2026-02-13T02:22:16.014565+0000","last_active":"2026-02-13T02:22:18.051246+0000","last_peered":"2026-02-13T02:22:18.051246+0000","last_clean":"2026-02-13T02:22:18.051246+0000","last_became_active":"2026-02-13T02:22:16.013815+0000","last_became_peered":"2026-02-13T02:22:16.013815+0000","last_unstale":"2026-02-13T02:22:18.051246+0000","last_undegraded":"2026-02-13T02:22:18.051246+0000","last_fullsized":"2026-02-13T02:22:18.051246+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T04:34:31.290868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047168+0000","last_change":"2026-02-13T02:22:17.013289+0000","last_active":"2026-02-13T02:22:18.047168+0000","last_peered":"2026-02-13T02:22:18.047168+0000","last_clean":"2026-02-13T02:22:18.047168+0000","last_became_active":"2026-02-13T02:22:17.013160+0000","last_became_peered":"2026-02-13T02:22:17.013160+0000","last_unstale":"2026-02-13T02:22:18.047168+0000","last_undegraded":"2026-02-13T02:22:18.047168+0000","last_fullsized":"2026-02-13T02:22:18.047168+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:19:06.942113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047197+0000","last_change":"2026-02-13T02:22:16.007420+0000","last_active":"2026-02-13T02:22:18.047197+0000","last_peered":"2026-02-13T02:22:18.047197+0000","last_clean":"2026-02-13T02:22:18.047197+0000","last_became_active":"2026-02-13T02:22:16.007114+0000","last_became_peered":"2026-02-13T02:22:16.007114+0000","last_unstale":"2026-02-13T02:22:18.047197+0000","last_undegraded":"2026-02-13T02:22:18.047197+0000","last_fullsized":"2026-02-13T02:22:18.047197+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:54:40.155679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051284+0000","last_change":"2026-02-13T02:22:17.021679+0000","last_active":"2026-02-13T02:22:18.051284+0000","last_peered":"2026-02-13T02:22:18.051284+0000","last_clean":"2026-02-13T02:22:18.051284+0000","last_became_active":"2026-02-13T02:22:17.021041+0000","last_became_peered":"2026-02-13T02:22:17.021041+0000","last_unstale":"2026-02-13T02:22:18.051284+0000","last_undegraded":"2026-02-13T02:22:18.051284+0000","last_fullsized":"2026-02-13T02:22:18.051284+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T04:37:08.611062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051417+0000","last_change":"2026-02-13T02:22:16.016749+0000","last_active":"2026-02-13T02:22:18.051417+0000","last_peered":"2026-02-13T02:22:18.051417+0000","last_clean":"2026-02-13T02:22:18.051417+0000","last_became_active":"2026-02-13T02:22:16.016620+0000","last_became_peered":"2026-02-13T02:22:16.016620+0000","last_unstale":"2026-02-13T02:22:18.051417+0000","last_undegraded":"2026-02-13T02:22:18.051417+0000","last_fullsized":"2026-02-13T02:22:18.051417+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:29:12.370221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051646+0000","last_change":"2026-02-13T02:22:17.025708+0000","last_active":"2026-02-13T02:22:18.051646+0000","last_peered":"2026-02-13T02:22:18.051646+0000","last_clean":"2026-02-13T02:22:18.051646+0000","last_became_active":"2026-02-13T02:22:17.025240+0000","last_became_peered":"2026-02-13T02:22:17.025240+0000","last_unstale":"2026-02-13T02:22:18.051646+0000","last_undegraded":"2026-02-13T02:22:18.051646+0000","last_fullsized":"2026-02-13T02:22:18.051646+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:04:02.085370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.011935+0000","last_change":"2026-02-13T02:22:16.009592+0000","last_active":"2026-02-13T02:22:18.011935+0000","last_peered":"2026-02-13T02:22:18.011935+0000","last_clean":"2026-02-13T02:22:18.011935+0000","last_became_active":"2026-02-13T02:22:16.009465+0000","last_became_peered":"2026-02-13T02:22:16.009465+0000","last_unstale":"2026-02-13T02:22:18.011935+0000","last_undegraded":"2026-02-13T02:22:18.011935+0000","last_fullsized":"2026-02-13T02:22:18.011935+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T02:33:29.133373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051794+0000","last_change":"2026-02-13T02:22:17.023729+0000","last_active":"2026-02-13T02:22:18.051794+0000","last_peered":"2026-02-13T02:22:18.051794+0000","last_clean":"2026-02-13T02:22:18.051794+0000","last_became_active":"2026-02-13T02:22:17.022007+0000","last_became_peered":"2026-02-13T02:22:17.022007+0000","last_unstale":"2026-02-13T02:22:18.051794+0000","last_undegraded":"2026-02-13T02:22:18.051794+0000","last_fullsized":"2026-02-13T02:22:18.051794+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T10:34:43.688805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.025077+0000","last_change":"2026-02-13T02:22:16.012023+0000","last_active":"2026-02-13T02:22:18.025077+0000","last_peered":"2026-02-13T02:22:18.025077+0000","last_clean":"2026-02-13T02:22:18.025077+0000","last_became_active":"2026-02-13T02:22:16.011680+0000","last_became_peered":"2026-02-13T02:22:16.011680+0000","last_unstale":"2026-02-13T02:22:18.025077+0000","last_undegraded":"2026-02-13T02:22:18.025077+0000","last_fullsized":"2026-02-13T02:22:18.025077+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T02:28:59.078471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047450+0000","last_change":"2026-02-13T02:22:17.015877+0000","last_active":"2026-02-13T02:22:18.047450+0000","last_peered":"2026-02-13T02:22:18.047450+0000","last_clean":"2026-02-13T02:22:18.047450+0000","last_became_active":"2026-02-13T02:22:17.015701+0000","last_became_peered":"2026-02-13T02:22:17.015701+0000","last_unstale":"2026-02-13T02:22:18.047450+0000","last_undegraded":"2026-02-13T02:22:18.047450+0000","last_fullsized":"2026-02-13T02:22:18.047450+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:35:45.903754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"70'1","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.020452+0000","last_change":"2026-02-13T02:22:16.008749+0000","last_active":"2026-02-13T02:22:18.020452+0000","last_peered":"2026-02-13T02:22:18.020452+0000","last_clean":"2026-02-13T02:22:18.020452+0000","last_became_active":"2026-02-13T02:22:16.008597+0000","last_became_peered":"2026-02-13T02:22:16.008597+0000","last_unstale":"2026-02-13T02:22:18.020452+0000","last_undegraded":"2026-02-13T02:22:18.020452+0000","last_fullsized":"2026-02-13T02:22:18.020452+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T10:18:25.963625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.021323+0000","last_change":"2026-02-13T02:22:17.017797+0000","last_active":"2026-02-13T02:22:18.021323+0000","last_peered":"2026-02-13T02:22:18.021323+0000","last_clean":"2026-02-13T02:22:18.021323+0000","last_became_active":"2026-02-13T02:22:17.017174+0000","last_became_peered":"2026-02-13T02:22:17.017174+0000","last_unstale":"2026-02-13T02:22:18.021323+0000","last_undegraded":"2026-02-13T02:22:18.021323+0000","last_fullsized":"2026-02-13T02:22:18.021323+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:39:41.102807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"69'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2026-02-13T02:22:17.026121+0000","last_change":"2026-02-13T02:22:16.017646+0000","last_active":"2026-02-13T02:22:17.026121+0000","last_peered":"2026-02-13T02:22:17.026121+0000","last_clean":"2026-02-13T02:22:17.026121+0000","last_became_active":"2026-02-13T02:22:16.017529+0000","last_became_peered":"2026-02-13T02:22:16.017529+0000","last_unstale":"2026-02-13T02:22:17.026121+0000","last_undegraded":"2026-02-13T02:22:17.026121+0000","last_fullsized":"2026-02-13T02:22:17.026121+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T12:34:48.977200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047594+0000","last_change":"2026-02-13T02:22:17.019503+0000","last_active":"2026-02-13T02:22:18.047594+0000","last_peered":"2026-02-13T02:22:18.047594+0000","last_clean":"2026-02-13T02:22:18.047594+0000","last_became_active":"2026-02-13T02:22:17.019259+0000","last_became_peered":"2026-02-13T02:22:17.019259+0000","last_unstale":"2026-02-13T02:22:18.047594+0000","last_undegraded":"2026-02-13T02:22:18.047594+0000","last_fullsized":"2026-02-13T02:22:18.047594+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:19:49.339245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.050971+0000","last_change":"2026-02-13T02:22:16.016248+0000","last_active":"2026-02-13T02:22:18.050971+0000","last_peered":"2026-02-13T02:22:18.050971+0000","last_clean":"2026-02-13T02:22:18.050971+0000","last_became_active":"2026-02-13T02:22:16.016016+0000","last_became_peered":"2026-02-13T02:22:16.016016+0000","last_unstale":"2026-02-13T02:22:18.050971+0000","last_undegraded":"2026-02-13T02:22:18.050971+0000","last_fullsized":"2026-02-13T02:22:18.050971+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:39:11.300311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.024643+0000","last_change":"2026-02-13T02:22:17.023656+0000","last_active":"2026-02-13T02:22:18.024643+0000","last_peered":"2026-02-13T02:22:18.024643+0000","last_clean":"2026-02-13T02:22:18.024643+0000","last_became_active":"2026-02-13T02:22:17.023524+0000","last_became_peered":"2026-02-13T02:22:17.023524+0000","last_unstale":"2026-02-13T02:22:18.024643+0000","last_undegraded":"2026-02-13T02:22:18.024643+0000","last_fullsized":"2026-02-13T02:22:18.024643+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:33:34.390446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.020934+0000","last_change":"2026-02-13T02:22:16.010000+0000","last_active":"2026-02-13T02:22:18.020934+0000","last_peered":"2026-02-13T02:22:18.020934+0000","last_clean":"2026-02-13T02:22:18.020934+0000","last_became_active":"2026-02-13T02:22:16.009736+0000","last_became_peered":"2026-02-13T02:22:16.009736+0000","last_unstale":"2026-02-13T02:22:18.020934+0000","last_undegraded":"2026-02-13T02:22:18.020934+0000","last_fullsized":"2026-02-13T02:22:18.020934+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:16:46.363188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016302+0000","last_change":"2026-02-13T02:22:17.022375+0000","last_active":"2026-02-13T02:22:18.016302+0000","last_peered":"2026-02-13T02:22:18.016302+0000","last_clean":"2026-02-13T02:22:18.016302+0000","last_became_active":"2026-02-13T02:22:17.022236+0000","last_became_peered":"2026-02-13T02:22:17.022236+0000","last_unstale":"2026-02-13T02:22:18.016302+0000","last_undegraded":"2026-02-13T02:22:18.016302+0000","last_fullsized":"2026-02-13T02:22:18.016302+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:32:39.302116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.015763+0000","last_change":"2026-02-13T02:22:16.015222+0000","last_active":"2026-02-13T02:22:18.015763+0000","last_peered":"2026-02-13T02:22:18.015763+0000","last_clean":"2026-02-13T02:22:18.015763+0000","last_became_active":"2026-02-13T02:22:16.014947+0000","last_became_peered":"2026-02-13T02:22:16.014947+0000","last_unstale":"2026-02-13T02:22:18.015763+0000","last_undegraded":"2026-02-13T02:22:18.015763+0000","last_fullsized":"2026-02-13T02:22:18.015763+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T14:01:02.677585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.021118+0000","last_change":"2026-02-13T02:22:17.021948+0000","last_active":"2026-02-13T02:22:18.021118+0000","last_peered":"2026-02-13T02:22:18.021118+0000","last_clean":"2026-02-13T02:22:18.021118+0000","last_became_active":"2026-02-13T02:22:17.021764+0000","last_became_peered":"2026-02-13T02:22:17.021764+0000","last_unstale":"2026-02-13T02:22:18.021118+0000","last_undegraded":"2026-02-13T02:22:18.021118+0000","last_fullsized":"2026-02-13T02:22:18.021118+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:39:50.788814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"69'2","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.024813+0000","last_change":"2026-02-13T02:22:16.014740+0000","last_active":"2026-02-13T02:22:18.024813+0000","last_peered":"2026-02-13T02:22:18.024813+0000","last_clean":"2026-02-13T02:22:18.024813+0000","last_became_active":"2026-02-13T02:22:16.014578+0000","last_became_peered":"2026-02-13T02:22:16.014578+0000","last_unstale":"2026-02-13T02:22:18.024813+0000","last_undegraded":"2026-02-13T02:22:18.024813+0000","last_fullsized":"2026-02-13T02:22:18.024813+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:09:10.619351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051553+0000","last_change":"2026-02-13T02:22:17.025211+0000","last_active":"2026-02-13T02:22:18.051553+0000","last_peered":"2026-02-13T02:22:18.051553+0000","last_clean":"2026-02-13T02:22:18.051553+0000","last_became_active":"2026-02-13T02:22:17.025015+0000","last_became_peered":"2026-02-13T02:22:17.025015+0000","last_unstale":"2026-02-13T02:22:18.051553+0000","last_undegraded":"2026-02-13T02:22:18.051553+0000","last_fullsized":"2026-02-13T02:22:18.051553+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T04:28:08.456016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051394+0000","last_change":"2026-02-13T02:22:17.022434+0000","last_active":"2026-02-13T02:22:18.051394+0000","last_peered":"2026-02-13T02:22:18.051394+0000","last_clean":"2026-02-13T02:22:18.051394+0000","last_became_active":"2026-02-13T02:22:17.022160+0000","last_became_peered":"2026-02-13T02:22:17.022160+0000","last_unstale":"2026-02-13T02:22:18.051394+0000","last_undegraded":"2026-02-13T02:22:18.051394+0000","last_fullsized":"2026-02-13T02:22:18.051394+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T12:27:17.994699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"69'2","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.024901+0000","last_change":"2026-02-13T02:22:16.013867+0000","last_active":"2026-02-13T02:22:18.024901+0000","last_peered":"2026-02-13T02:22:18.024901+0000","last_clean":"2026-02-13T02:22:18.024901+0000","last_became_active":"2026-02-13T02:22:16.013689+0000","last_became_peered":"2026-02-13T02:22:16.013689+0000","last_unstale":"2026-02-13T02:22:18.024901+0000","last_undegraded":"2026-02-13T02:22:18.024901+0000","last_fullsized":"2026-02-13T02:22:18.024901+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T04:46:39.476015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051182+0000","last_change":"2026-02-13T02:22:17.022407+0000","last_active":"2026-02-13T02:22:18.051182+0000","last_peered":"2026-02-13T02:22:18.051182+0000","last_clean":"2026-02-13T02:22:18.051182+0000","last_became_active":"2026-02-13T02:22:17.021354+0000","last_became_peered":"2026-02-13T02:22:17.021354+0000","last_unstale":"2026-02-13T02:22:18.051182+0000","last_undegraded":"2026-02-13T02:22:18.051182+0000","last_fullsized":"2026-02-13T02:22:18.051182+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:35:18.694829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"69'2","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.025017+0000","last_change":"2026-02-13T02:22:16.015538+0000","last_active":"2026-02-13T02:22:18.025017+0000","last_peered":"2026-02-13T02:22:18.025017+0000","last_clean":"2026-02-13T02:22:18.025017+0000","last_became_active":"2026-02-13T02:22:16.015441+0000","last_became_peered":"2026-02-13T02:22:16.015441+0000","last_unstale":"2026-02-13T02:22:18.025017+0000","last_undegraded":"2026-02-13T02:22:18.025017+0000","last_fullsized":"2026-02-13T02:22:18.025017+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T02:34:39.849666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047574+0000","last_change":"2026-02-13T02:22:17.018483+0000","last_active":"2026-02-13T02:22:18.047574+0000","last_peered":"2026-02-13T02:22:18.047574+0000","last_clean":"2026-02-13T02:22:18.047574+0000","last_became_active":"2026-02-13T02:22:17.018318+0000","last_became_peered":"2026-02-13T02:22:17.018318+0000","last_unstale":"2026-02-13T02:22:18.047574+0000","last_undegraded":"2026-02-13T02:22:18.047574+0000","last_fullsized":"2026-02-13T02:22:18.047574+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T10:22:49.603621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"69'1","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.021277+0000","last_change":"2026-02-13T02:22:16.008795+0000","last_active":"2026-02-13T02:22:18.021277+0000","last_peered":"2026-02-13T02:22:18.021277+0000","last_clean":"2026-02-13T02:22:18.021277+0000","last_became_active":"2026-02-13T02:22:16.008621+0000","last_became_peered":"2026-02-13T02:22:16.008621+0000","last_unstale":"2026-02-13T02:22:18.021277+0000","last_undegraded":"2026-02-13T02:22:18.021277+0000","last_fullsized":"2026-02-13T02:22:18.021277+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:43:45.861077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.021240+0000","last_change":"2026-02-13T02:22:17.019826+0000","last_active":"2026-02-13T02:22:18.021240+0000","last_peered":"2026-02-13T02:22:18.021240+0000","last_clean":"2026-02-13T02:22:18.021240+0000","last_became_active":"2026-02-13T02:22:17.019429+0000","last_became_peered":"2026-02-13T02:22:17.019429+0000","last_unstale":"2026-02-13T02:22:18.021240+0000","last_undegraded":"2026-02-13T02:22:18.021240+0000","last_fullsized":"2026-02-13T02:22:18.021240+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:52:59.597088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"69'4","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.012046+0000","last_change":"2026-02-13T02:22:16.011002+0000","last_active":"2026-02-13T02:22:18.012046+0000","last_peered":"2026-02-13T02:22:18.012046+0000","last_clean":"2026-02-13T02:22:18.012046+0000","last_became_active":"2026-02-13T02:22:16.010892+0000","last_became_peered":"2026-02-13T02:22:16.010892+0000","last_unstale":"2026-02-13T02:22:18.012046+0000","last_undegraded":"2026-02-13T02:22:18.012046+0000","last_fullsized":"2026-02-13T02:22:18.012046+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:57:22.333777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":604,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":9205,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":9,"num_read_kb":7,"num_write":48,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":16582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":65369,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":189,"num_write_kb":3716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":264161,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":65,"seq":279172874245,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27696,"kb_used_data":732,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":733975504,"statfs":{"total":751619276800,"available":751590916096,"internally_reserved":0,"allocated":749568,"data_stored":364081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8466,"internal_metadata":27582190},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":58,"seq":249108103175,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27768,"kb_used_data":804,"kb_used_omap":18,"kb_used_meta":26925,"kb_avail":733975432,"statfs":{"total":751619276800,"available":751590842368,"internally_reserved":0,"allocated":823296,"data_stored":956277,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":19356,"internal_metadata":27571300},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":54,"seq":231928233993,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27700,"kb_used_data":736,"kb_used_omap":12,"kb_used_meta":26931,"kb_avail":733975500,"statfs":{"total":751619276800,"available":751590912000,"internally_reserved":0,"allocated":753664,"data_stored":364685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12485,"internal_metadata":27578171},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":47,"seq":201863462922,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27704,"kb_used_data":740,"kb_used_omap":20,"kb_used_meta":26923,"kb_avail":733975496,"statfs":{"total":751619276800,"available":751590907904,"internally_reserved":0,"allocated":757760,"data_stored":364707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21090,"internal_metadata":27569566},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":40,"seq":171798691851,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27748,"kb_used_data":784,"kb_used_omap":23,"kb_used_meta":26920,"kb_avail":733975452,"statfs":{"total":751619276800,"available":751590862848,"internally_reserved":0,"allocated":802816,"data_stored":955053,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":24500,"internal_metadata":27566156},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":37,"seq":158913789966,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27696,"kb_used_data":732,"kb_used_omap":23,"kb_used_meta":26920,"kb_avail":733975504,"statfs":{"total":751619276800,"available":751590916096,"internally_reserved":0,"allocated":749568,"data_stored":364081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24460,"internal_metadata":27566196},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":30,"seq":128849018895,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27756,"kb_used_data":792,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":733975444,"statfs":{"total":751619276800,"available":751590854656,"internally_reserved":0,"allocated":811008,"data_stored":954631,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":29636,"internal_metadata":27561020},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":26,"seq":111669149712,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27688,"kb_used_data":724,"kb_used_omap":23,"kb_used_meta":26920,"kb_avail":733975512,"statfs":{"total":751619276800,"available":751590924288,"internally_reserved":0,"allocated":741376,"data_stored":355747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24232,"internal_metadata":27566424},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":19,"seq":81604378642,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27704,"kb_used_data":740,"kb_used_omap":39,"kb_used_meta":26904,"kb_avail":733975496,"statfs":{"total":751619276800,"available":751590907904,"internally_reserved":0,"allocated":757760,"data_stored":364185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":40627,"internal_metadata":27550029},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27712,"kb_used_data":748,"kb_used_omap":48,"kb_used_meta":26895,"kb_avail":733975488,"statfs":{"total":751619276800,"available":751590899712,"internally_reserved":0,"allocated":765952,"data_stored":365749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49734,"internal_metadata":27540922},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672981,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27708,"kb_used_data":744,"kb_used_omap":38,"kb_used_meta":26905,"kb_avail":733975492,"statfs":{"total":751619276800,"available":751590903808,"internally_reserved":0,"allocated":761856,"data_stored":365715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39406,"internal_metadata":27551250},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":7,"seq":30064771094,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27700,"kb_used_data":736,"kb_used_omap":40,"kb_used_meta":26903,"kb_avail":733975500,"statfs":{"total":751619276800,"available":751590912000,"internally_reserved":0,"allocated":753664,"data_stored":364171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41083,"internal_metadata":27549573},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":36563,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":36563,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20097,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":4103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":705,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13826,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":426,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1256,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":366,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1586,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":656,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4234,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":470,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":8579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17757,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3990,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-13T02:22:24.654 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2026-02-13T02:22:24.655 INFO:tasks.cephadm:Setting up client nodes... 2026-02-13T02:22:24.655 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2026-02-13T02:22:24.771 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:24.896 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.893+0000 7fc3f26416c0 1 -- 10.20.193.27:0/4178082216 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc3ec10eed0 msgr2=0x7fc3ec10f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:24.897 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.893+0000 7fc3f26416c0 1 --2- 10.20.193.27:0/4178082216 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc3ec10eed0 0x7fc3ec10f2c0 secure :-1 s=READY pgs=249 gs=1 cs=0 l=1 c_cookie=3006744607615624573 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc3d8009970 tx=0x7fc3d802ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:24.897 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.893+0000 7fc3f26416c0 1 -- 10.20.193.27:0/4178082216 shutdown_connections 2026-02-13T02:22:24.897 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.893+0000 7fc3f26416c0 1 --2- 10.20.193.27:0/4178082216 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc3ec110a00 0x7fc3ec171390 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:24.897 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.893+0000 7fc3f26416c0 1 --2- 10.20.193.27:0/4178082216 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc3ec110050 0x7fc3ec1104c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:24.897 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.893+0000 7fc3f26416c0 1 --2- 10.20.193.27:0/4178082216 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc3ec10eed0 0x7fc3ec10f2c0 unknown :-1 s=CLOSED pgs=249 gs=1 cs=0 l=1 c_cookie=3006744607615624573 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:24.897 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.893+0000 7fc3f26416c0 1 -- 10.20.193.27:0/4178082216 >> 10.20.193.27:0/4178082216 conn(0x7fc3ec0ec5f0 msgr2=0x7fc3ec10ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:24.897 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.893+0000 7fc3f26416c0 1 -- 10.20.193.27:0/4178082216 shutdown_connections 2026-02-13T02:22:24.897 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.893+0000 7fc3f26416c0 1 -- 10.20.193.27:0/4178082216 wait complete. 2026-02-13T02:22:24.897 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.893+0000 7fc3f26416c0 1 Processor -- start 2026-02-13T02:22:24.897 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.893+0000 7fc3f26416c0 1 -- start start 2026-02-13T02:22:24.898 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.893+0000 7fc3f26416c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc3ec10eed0 0x7fc3ec0f85e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:24.898 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.893+0000 7fc3f26416c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc3ec110050 0x7fc3ec171390 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:24.898 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.893+0000 7fc3f26416c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc3ec110a00 0x7fc3ec0f8e40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:24.898 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.893+0000 7fc3f26416c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec0f9a70 con 0x7fc3ec110a00 2026-02-13T02:22:24.898 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.893+0000 7fc3f26416c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec0f9be0 con 0x7fc3ec10eed0 2026-02-13T02:22:24.898 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.893+0000 7fc3f26416c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec0f9da0 con 0x7fc3ec110050 2026-02-13T02:22:24.898 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.893+0000 7fc3f0bb36c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc3ec110a00 0x7fc3ec0f8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:24.898 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.893+0000 7fc3f0bb36c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc3ec110a00 0x7fc3ec0f8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55454/0 (socket says 10.20.193.27:55454) 2026-02-13T02:22:24.898 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.893+0000 7fc3f0bb36c0 1 -- 10.20.193.27:0/1479146272 learned_addr learned my addr 10.20.193.27:0/1479146272 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:24.898 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.893+0000 7fc3ebfff6c0 1 --2- 10.20.193.27:0/1479146272 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc3ec10eed0 0x7fc3ec0f85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:24.898 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.893+0000 7fc3eb7fe6c0 1 --2- 10.20.193.27:0/1479146272 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc3ec110050 0x7fc3ec171390 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:24.899 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.894+0000 7fc3f0bb36c0 1 -- 10.20.193.27:0/1479146272 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc3ec110050 msgr2=0x7fc3ec171390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:24.899 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.894+0000 7fc3f0bb36c0 1 --2- 10.20.193.27:0/1479146272 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc3ec110050 0x7fc3ec171390 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:24.899 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.894+0000 7fc3f0bb36c0 1 -- 10.20.193.27:0/1479146272 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc3ec10eed0 msgr2=0x7fc3ec0f85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:24.899 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.894+0000 7fc3f0bb36c0 1 --2- 10.20.193.27:0/1479146272 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc3ec10eed0 0x7fc3ec0f85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:24.899 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.894+0000 7fc3f0bb36c0 1 -- 10.20.193.27:0/1479146272 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d8009590 con 0x7fc3ec110a00 2026-02-13T02:22:24.899 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.894+0000 7fc3f0bb36c0 1 --2- 10.20.193.27:0/1479146272 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc3ec110a00 0x7fc3ec0f8e40 secure :-1 s=READY pgs=250 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc3e0006ea0 tx=0x7fc3e000ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:24.899 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.894+0000 7fc3e97fa6c0 1 -- 10.20.193.27:0/1479146272 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc3e0003c40 con 0x7fc3ec110a00 2026-02-13T02:22:24.899 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.894+0000 7fc3e97fa6c0 1 -- 10.20.193.27:0/1479146272 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc3e0003da0 con 0x7fc3ec110a00 2026-02-13T02:22:24.899 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.894+0000 7fc3f26416c0 1 -- 10.20.193.27:0/1479146272 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3ec0fa080 con 0x7fc3ec110a00 2026-02-13T02:22:24.899 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.894+0000 7fc3e97fa6c0 1 -- 10.20.193.27:0/1479146272 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc3e00081c0 con 0x7fc3ec110a00 2026-02-13T02:22:24.899 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.894+0000 7fc3f26416c0 1 -- 10.20.193.27:0/1479146272 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3ec0fa5b0 con 0x7fc3ec110a00 2026-02-13T02:22:24.899 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.894+0000 7fc3f26416c0 1 -- 10.20.193.27:0/1479146272 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3ec175270 con 0x7fc3ec110a00 2026-02-13T02:22:24.899 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.895+0000 7fc3e97fa6c0 1 -- 10.20.193.27:0/1479146272 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7fc3e0005860 con 0x7fc3ec110a00 2026-02-13T02:22:24.901 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.896+0000 7fc3e97fa6c0 1 --2- 10.20.193.27:0/1479146272 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc3c00b5710 0x7fc3c00d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:24.901 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.896+0000 7fc3e97fa6c0 1 -- 10.20.193.27:0/1479146272 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7fc3e00e4b10 con 0x7fc3ec110a00 2026-02-13T02:22:24.901 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.896+0000 7fc3ebfff6c0 1 --2- 10.20.193.27:0/1479146272 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc3c00b5710 0x7fc3c00d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:24.901 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.896+0000 7fc3ebfff6c0 1 --2- 10.20.193.27:0/1479146272 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc3c00b5710 0x7fc3c00d5bc0 secure :-1 s=READY pgs=180 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc3d80023e0 tx=0x7fc3d8005c00 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:24.901 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:24.896+0000 7fc3e97fa6c0 1 -- 10.20.193.27:0/1479146272 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fc3e00a7570 con 0x7fc3ec110a00 2026-02-13T02:22:25.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.024+0000 7fc3f26416c0 1 -- 10.20.193.27:0/1479146272 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fc3ec10f2a0 con 0x7fc3ec110a00 2026-02-13T02:22:25.027 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.025+0000 7fc3e97fa6c0 1 -- 10.20.193.27:0/1479146272 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) ==== 170+0+59 (secure 0 0 0) 0x7fc3e00aa720 con 0x7fc3ec110a00 2026-02-13T02:22:25.027 INFO:teuthology.orchestra.run.trial027.stdout:[client.0] 2026-02-13T02:22:25.028 INFO:teuthology.orchestra.run.trial027.stdout: key = AQDhio5pFlZ/ARAAX9FvjPjyS7tdT1m/g2WpCQ== 2026-02-13T02:22:25.029 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.026+0000 7fc3f26416c0 1 -- 10.20.193.27:0/1479146272 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc3c00b5710 msgr2=0x7fc3c00d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:25.029 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.026+0000 7fc3f26416c0 1 --2- 10.20.193.27:0/1479146272 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc3c00b5710 0x7fc3c00d5bc0 secure :-1 s=READY pgs=180 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc3d80023e0 tx=0x7fc3d8005c00 comp rx=0 tx=0).stop 2026-02-13T02:22:25.029 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.026+0000 7fc3f26416c0 1 -- 10.20.193.27:0/1479146272 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc3ec110a00 msgr2=0x7fc3ec0f8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:25.029 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.026+0000 7fc3f26416c0 1 --2- 10.20.193.27:0/1479146272 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc3ec110a00 0x7fc3ec0f8e40 secure :-1 s=READY pgs=250 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc3e0006ea0 tx=0x7fc3e000ce50 comp rx=0 tx=0).stop 2026-02-13T02:22:25.029 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.026+0000 7fc3f26416c0 1 -- 10.20.193.27:0/1479146272 shutdown_connections 2026-02-13T02:22:25.029 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.026+0000 7fc3f26416c0 1 --2- 10.20.193.27:0/1479146272 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc3c00b5710 0x7fc3c00d5bc0 unknown :-1 s=CLOSED pgs=180 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:25.029 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.026+0000 7fc3f26416c0 1 --2- 10.20.193.27:0/1479146272 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc3ec110a00 0x7fc3ec0f8e40 unknown :-1 s=CLOSED pgs=250 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:25.029 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.026+0000 7fc3f26416c0 1 --2- 10.20.193.27:0/1479146272 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc3ec110050 0x7fc3ec171390 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:25.029 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.026+0000 7fc3f26416c0 1 --2- 10.20.193.27:0/1479146272 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc3ec10eed0 0x7fc3ec0f85e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:25.029 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.026+0000 7fc3f26416c0 1 -- 10.20.193.27:0/1479146272 >> 10.20.193.27:0/1479146272 conn(0x7fc3ec0ec5f0 msgr2=0x7fc3ec0eeb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:25.030 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.026+0000 7fc3f26416c0 1 -- 10.20.193.27:0/1479146272 shutdown_connections 2026-02-13T02:22:25.030 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.026+0000 7fc3f26416c0 1 -- 10.20.193.27:0/1479146272 wait complete. 2026-02-13T02:22:25.069 DEBUG:teuthology.orchestra.run.trial027:> set -ex 2026-02-13T02:22:25.069 DEBUG:teuthology.orchestra.run.trial027:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2026-02-13T02:22:25.069 DEBUG:teuthology.orchestra.run.trial027:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-13T02:22:25.104 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph config log 1 --format=json 2026-02-13T02:22:25.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:25 trial027 ceph-mon[24567]: from='client.14859 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-13T02:22:25.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:25 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1479146272' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2026-02-13T02:22:25.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:25 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1479146272' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2026-02-13T02:22:25.258 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:25.386 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.382+0000 7fe584d8c6c0 1 -- 10.20.193.27:0/4132605797 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe58010fbf0 msgr2=0x7fe580114c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:25.386 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.382+0000 7fe584d8c6c0 1 --2- 10.20.193.27:0/4132605797 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe58010fbf0 0x7fe580114c30 secure :-1 s=READY pgs=251 gs=1 cs=0 l=1 c_cookie=13698769616957432693 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe5680099a0 tx=0x7fe56802ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:25.386 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.382+0000 7fe584d8c6c0 1 -- 10.20.193.27:0/4132605797 shutdown_connections 2026-02-13T02:22:25.386 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.382+0000 7fe584d8c6c0 1 --2- 10.20.193.27:0/4132605797 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe580115410 0x7fe580175db0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:25.386 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.382+0000 7fe584d8c6c0 1 --2- 10.20.193.27:0/4132605797 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe58010fbf0 0x7fe580114c30 unknown :-1 s=CLOSED pgs=251 gs=1 cs=0 l=1 c_cookie=13698769616957432693 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:25.386 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.382+0000 7fe584d8c6c0 1 --2- 10.20.193.27:0/4132605797 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe58010f200 0x7fe58010f5d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:25.387 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.382+0000 7fe584d8c6c0 1 -- 10.20.193.27:0/4132605797 >> 10.20.193.27:0/4132605797 conn(0x7fe5800ecb00 msgr2=0x7fe58010cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:25.387 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.382+0000 7fe584d8c6c0 1 -- 10.20.193.27:0/4132605797 shutdown_connections 2026-02-13T02:22:25.387 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.382+0000 7fe584d8c6c0 1 -- 10.20.193.27:0/4132605797 wait complete. 2026-02-13T02:22:25.387 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.382+0000 7fe584d8c6c0 1 Processor -- start 2026-02-13T02:22:25.387 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.382+0000 7fe584d8c6c0 1 -- start start 2026-02-13T02:22:25.387 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.383+0000 7fe584d8c6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe58010f200 0x7fe5800f4d50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:25.387 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.383+0000 7fe584d8c6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe580115410 0x7fe580175cb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:25.387 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.383+0000 7fe584d8c6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe5800f5380 0x7fe5800fa650 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:25.387 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.383+0000 7fe584d8c6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe5800f5a70 con 0x7fe5800f5380 2026-02-13T02:22:25.387 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.383+0000 7fe584d8c6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe5800fab90 con 0x7fe580115410 2026-02-13T02:22:25.387 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.383+0000 7fe584d8c6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe5800fad50 con 0x7fe58010f200 2026-02-13T02:22:25.387 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.383+0000 7fe57ed766c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe5800f5380 0x7fe5800fa650 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:25.388 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.383+0000 7fe57ed766c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe5800f5380 0x7fe5800fa650 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55474/0 (socket says 10.20.193.27:55474) 2026-02-13T02:22:25.388 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.383+0000 7fe57ed766c0 1 -- 10.20.193.27:0/3655885843 learned_addr learned my addr 10.20.193.27:0/3655885843 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:25.388 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.383+0000 7fe57dd746c0 1 --2- 10.20.193.27:0/3655885843 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe580115410 0x7fe580175cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:25.388 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.383+0000 7fe57e5756c0 1 --2- 10.20.193.27:0/3655885843 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe58010f200 0x7fe5800f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:25.388 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.383+0000 7fe57ed766c0 1 -- 10.20.193.27:0/3655885843 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe58010f200 msgr2=0x7fe5800f4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:25.388 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.383+0000 7fe57ed766c0 1 --2- 10.20.193.27:0/3655885843 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe58010f200 0x7fe5800f4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:25.388 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.383+0000 7fe57ed766c0 1 -- 10.20.193.27:0/3655885843 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe580115410 msgr2=0x7fe580175cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:25.388 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.383+0000 7fe57ed766c0 1 --2- 10.20.193.27:0/3655885843 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe580115410 0x7fe580175cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:25.388 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.383+0000 7fe57ed766c0 1 -- 10.20.193.27:0/3655885843 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe568009590 con 0x7fe5800f5380 2026-02-13T02:22:25.388 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.383+0000 7fe57e5756c0 1 --2- 10.20.193.27:0/3655885843 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe58010f200 0x7fe5800f4d50 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:22:25.388 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.383+0000 7fe57ed766c0 1 --2- 10.20.193.27:0/3655885843 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe5800f5380 0x7fe5800fa650 secure :-1 s=READY pgs=252 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe570006ea0 tx=0x7fe57000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:25.389 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.383+0000 7fe5677fe6c0 1 -- 10.20.193.27:0/3655885843 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe570003a30 con 0x7fe5800f5380 2026-02-13T02:22:25.389 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.383+0000 7fe5677fe6c0 1 -- 10.20.193.27:0/3655885843 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fe5700054f0 con 0x7fe5800f5380 2026-02-13T02:22:25.389 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.383+0000 7fe584d8c6c0 1 -- 10.20.193.27:0/3655885843 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5800fb030 con 0x7fe5800f5380 2026-02-13T02:22:25.389 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.383+0000 7fe5677fe6c0 1 -- 10.20.193.27:0/3655885843 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe57000dcd0 con 0x7fe5800f5380 2026-02-13T02:22:25.389 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.383+0000 7fe584d8c6c0 1 -- 10.20.193.27:0/3655885843 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5800fb7c0 con 0x7fe5800f5380 2026-02-13T02:22:25.389 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.383+0000 7fe584d8c6c0 1 -- 10.20.193.27:0/3655885843 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5801104d0 con 0x7fe5800f5380 2026-02-13T02:22:25.389 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.385+0000 7fe5677fe6c0 1 -- 10.20.193.27:0/3655885843 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7fe570003b90 con 0x7fe5800f5380 2026-02-13T02:22:25.390 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.385+0000 7fe5677fe6c0 1 --2- 10.20.193.27:0/3655885843 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe5580b5710 0x7fe5580d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:25.390 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.385+0000 7fe57e5756c0 1 --2- 10.20.193.27:0/3655885843 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe5580b5710 0x7fe5580d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:25.390 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.385+0000 7fe5677fe6c0 1 -- 10.20.193.27:0/3655885843 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7fe5700e4a80 con 0x7fe5800f5380 2026-02-13T02:22:25.390 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.385+0000 7fe57e5756c0 1 --2- 10.20.193.27:0/3655885843 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe5580b5710 0x7fe5580d5bc0 secure :-1 s=READY pgs=181 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe574005980 tx=0x7fe574005df0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:25.390 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.386+0000 7fe5677fe6c0 1 -- 10.20.193.27:0/3655885843 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fe5700a74a0 con 0x7fe5800f5380 2026-02-13T02:22:25.477 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.476+0000 7fe584d8c6c0 1 -- 10.20.193.27:0/3655885843 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fe5800f5ea0 con 0x7fe5800f5380 2026-02-13T02:22:25.478 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.476+0000 7fe5677fe6c0 1 -- 10.20.193.27:0/3655885843 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v45) ==== 86+0+133 (secure 0 0 0) 0x7fe57000de60 con 0x7fe5800f5380 2026-02-13T02:22:25.478 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:22:25.479 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.478+0000 7fe584d8c6c0 1 -- 10.20.193.27:0/3655885843 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe5580b5710 msgr2=0x7fe5580d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:25.479 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.478+0000 7fe584d8c6c0 1 --2- 10.20.193.27:0/3655885843 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe5580b5710 0x7fe5580d5bc0 secure :-1 s=READY pgs=181 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe574005980 tx=0x7fe574005df0 comp rx=0 tx=0).stop 2026-02-13T02:22:25.479 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.478+0000 7fe584d8c6c0 1 -- 10.20.193.27:0/3655885843 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe5800f5380 msgr2=0x7fe5800fa650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:25.479 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.478+0000 7fe584d8c6c0 1 --2- 10.20.193.27:0/3655885843 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe5800f5380 0x7fe5800fa650 secure :-1 s=READY pgs=252 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe570006ea0 tx=0x7fe57000cca0 comp rx=0 tx=0).stop 2026-02-13T02:22:25.480 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.478+0000 7fe584d8c6c0 1 -- 10.20.193.27:0/3655885843 shutdown_connections 2026-02-13T02:22:25.480 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.478+0000 7fe584d8c6c0 1 --2- 10.20.193.27:0/3655885843 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe5580b5710 0x7fe5580d5bc0 unknown :-1 s=CLOSED pgs=181 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:25.480 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.478+0000 7fe584d8c6c0 1 --2- 10.20.193.27:0/3655885843 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe5800f5380 0x7fe5800fa650 unknown :-1 s=CLOSED pgs=252 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:25.480 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.478+0000 7fe584d8c6c0 1 --2- 10.20.193.27:0/3655885843 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe580115410 0x7fe580175cb0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:25.480 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.478+0000 7fe584d8c6c0 1 --2- 10.20.193.27:0/3655885843 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe58010f200 0x7fe5800f4d50 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:25.480 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.478+0000 7fe584d8c6c0 1 -- 10.20.193.27:0/3655885843 >> 10.20.193.27:0/3655885843 conn(0x7fe5800ecb00 msgr2=0x7fe580111c90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:25.480 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.478+0000 7fe584d8c6c0 1 -- 10.20.193.27:0/3655885843 shutdown_connections 2026-02-13T02:22:25.480 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.478+0000 7fe584d8c6c0 1 -- 10.20.193.27:0/3655885843 wait complete. 2026-02-13T02:22:25.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:25 trial028 ceph-mon[26701]: from='client.14859 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-13T02:22:25.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:25 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1479146272' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2026-02-13T02:22:25.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:25 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1479146272' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2026-02-13T02:22:25.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:25 trial107 ceph-mon[27422]: from='client.14859 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-13T02:22:25.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:25 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1479146272' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2026-02-13T02:22:25.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:25 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1479146272' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2026-02-13T02:22:25.519 INFO:teuthology.orchestra.run.trial027.stdout:[{"version":45,"timestamp":"2026-02-13T02:22:11.328241+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2026-02-13T02:22:25.519 INFO:tasks.ceph_manager:config epoch is 45 2026-02-13T02:22:25.519 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-13T02:22:25.519 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2026-02-13T02:22:25.520 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph mgr dump --format=json 2026-02-13T02:22:25.677 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:25.791 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.788+0000 7ff8b750d6c0 1 -- 10.20.193.27:0/1006930079 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff8b00669a0 msgr2=0x7ff8b00629b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:25.791 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.788+0000 7ff8b750d6c0 1 --2- 10.20.193.27:0/1006930079 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff8b00669a0 0x7ff8b00629b0 secure :-1 s=READY pgs=253 gs=1 cs=0 l=1 c_cookie=10353612773945423791 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff8a0009970 tx=0x7ff8a002ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:25.791 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.788+0000 7ff8b750d6c0 1 -- 10.20.193.27:0/1006930079 shutdown_connections 2026-02-13T02:22:25.792 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.788+0000 7ff8b750d6c0 1 --2- 10.20.193.27:0/1006930079 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff8b0159a50 0x7ff8b0179eb0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:25.792 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.788+0000 7ff8b750d6c0 1 --2- 10.20.193.27:0/1006930079 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff8b0062fd0 0x7ff8b0063400 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:25.792 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.788+0000 7ff8b750d6c0 1 --2- 10.20.193.27:0/1006930079 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff8b00669a0 0x7ff8b00629b0 unknown :-1 s=CLOSED pgs=253 gs=1 cs=0 l=1 c_cookie=10353612773945423791 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:25.792 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.788+0000 7ff8b750d6c0 1 -- 10.20.193.27:0/1006930079 >> 10.20.193.27:0/1006930079 conn(0x7ff8b00ecb00 msgr2=0x7ff8b010cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:25.792 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.788+0000 7ff8b750d6c0 1 -- 10.20.193.27:0/1006930079 shutdown_connections 2026-02-13T02:22:25.792 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.788+0000 7ff8b750d6c0 1 -- 10.20.193.27:0/1006930079 wait complete. 2026-02-13T02:22:25.792 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.788+0000 7ff8b750d6c0 1 Processor -- start 2026-02-13T02:22:25.792 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.788+0000 7ff8b750d6c0 1 -- start start 2026-02-13T02:22:25.792 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.788+0000 7ff8b750d6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff8b0062fd0 0x7ff8b01633c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:25.792 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.788+0000 7ff8b750d6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff8b0159a50 0x7ff8b0163900 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:25.792 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.788+0000 7ff8b750d6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff8b0168cc0 0x7ff8b0169160 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:25.793 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.788+0000 7ff8b750d6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b01696d0 con 0x7ff8b0159a50 2026-02-13T02:22:25.793 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.788+0000 7ff8b750d6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b0169840 con 0x7ff8b0168cc0 2026-02-13T02:22:25.793 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.788+0000 7ff8b750d6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b0169a00 con 0x7ff8b0062fd0 2026-02-13T02:22:25.793 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.788+0000 7ff8b4a7d6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff8b0159a50 0x7ff8b0163900 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:25.793 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.788+0000 7ff8b4a7d6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff8b0159a50 0x7ff8b0163900 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55486/0 (socket says 10.20.193.27:55486) 2026-02-13T02:22:25.793 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.788+0000 7ff8b4a7d6c0 1 -- 10.20.193.27:0/557174559 learned_addr learned my addr 10.20.193.27:0/557174559 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:25.793 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.788+0000 7ff8b527e6c0 1 --2- 10.20.193.27:0/557174559 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff8b0062fd0 0x7ff8b01633c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:25.793 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.788+0000 7ff8b5a7f6c0 1 --2- 10.20.193.27:0/557174559 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff8b0168cc0 0x7ff8b0169160 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:25.793 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.788+0000 7ff8b4a7d6c0 1 -- 10.20.193.27:0/557174559 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff8b0062fd0 msgr2=0x7ff8b01633c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:25.793 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.789+0000 7ff8b4a7d6c0 1 --2- 10.20.193.27:0/557174559 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff8b0062fd0 0x7ff8b01633c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:25.793 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.789+0000 7ff8b4a7d6c0 1 -- 10.20.193.27:0/557174559 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff8b0168cc0 msgr2=0x7ff8b0169160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:25.793 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.789+0000 7ff8b4a7d6c0 1 --2- 10.20.193.27:0/557174559 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff8b0168cc0 0x7ff8b0169160 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:25.794 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.789+0000 7ff8b4a7d6c0 1 -- 10.20.193.27:0/557174559 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a0009590 con 0x7ff8b0159a50 2026-02-13T02:22:25.794 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.789+0000 7ff8b4a7d6c0 1 --2- 10.20.193.27:0/557174559 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff8b0159a50 0x7ff8b0163900 secure :-1 s=READY pgs=254 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff8a40092e0 tx=0x7ff8a400bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:25.794 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.789+0000 7ff89e7fc6c0 1 -- 10.20.193.27:0/557174559 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff8a4004250 con 0x7ff8b0159a50 2026-02-13T02:22:25.794 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.789+0000 7ff89e7fc6c0 1 -- 10.20.193.27:0/557174559 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff8a40043b0 con 0x7ff8b0159a50 2026-02-13T02:22:25.794 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.789+0000 7ff8b750d6c0 1 -- 10.20.193.27:0/557174559 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b0157910 con 0x7ff8b0159a50 2026-02-13T02:22:25.794 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.789+0000 7ff89e7fc6c0 1 -- 10.20.193.27:0/557174559 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff8a400cd20 con 0x7ff8b0159a50 2026-02-13T02:22:25.794 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.789+0000 7ff8b750d6c0 1 -- 10.20.193.27:0/557174559 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b0157df0 con 0x7ff8b0159a50 2026-02-13T02:22:25.794 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.789+0000 7ff8b750d6c0 1 -- 10.20.193.27:0/557174559 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8b0063ec0 con 0x7ff8b0159a50 2026-02-13T02:22:25.795 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.790+0000 7ff89e7fc6c0 1 -- 10.20.193.27:0/557174559 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7ff8a40026b0 con 0x7ff8b0159a50 2026-02-13T02:22:25.795 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.791+0000 7ff89e7fc6c0 1 --2- 10.20.193.27:0/557174559 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff88c0b5710 0x7ff88c0d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:25.796 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.791+0000 7ff8b527e6c0 1 --2- 10.20.193.27:0/557174559 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff88c0b5710 0x7ff88c0d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:25.796 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.791+0000 7ff89e7fc6c0 1 -- 10.20.193.27:0/557174559 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7ff8a40e3ac0 con 0x7ff8b0159a50 2026-02-13T02:22:25.796 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.791+0000 7ff8b527e6c0 1 --2- 10.20.193.27:0/557174559 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff88c0b5710 0x7ff88c0d5bc0 secure :-1 s=READY pgs=182 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff8a00023e0 tx=0x7ff8a0005c60 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:25.796 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.791+0000 7ff89e7fc6c0 1 -- 10.20.193.27:0/557174559 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7ff8a40a9f40 con 0x7ff8b0159a50 2026-02-13T02:22:25.910 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.908+0000 7ff8b750d6c0 1 -- 10.20.193.27:0/557174559 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7ff8b0062980 con 0x7ff8b0159a50 2026-02-13T02:22:25.912 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.911+0000 7ff89e7fc6c0 1 -- 10.20.193.27:0/557174559 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v12) ==== 74+0+297579 (secure 0 0 0) 0x7ff8a40a9690 con 0x7ff8b0159a50 2026-02-13T02:22:25.917 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:22:25.920 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.913+0000 7ff8b750d6c0 1 -- 10.20.193.27:0/557174559 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff88c0b5710 msgr2=0x7ff88c0d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:25.920 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.913+0000 7ff8b750d6c0 1 --2- 10.20.193.27:0/557174559 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff88c0b5710 0x7ff88c0d5bc0 secure :-1 s=READY pgs=182 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff8a00023e0 tx=0x7ff8a0005c60 comp rx=0 tx=0).stop 2026-02-13T02:22:25.920 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.913+0000 7ff8b750d6c0 1 -- 10.20.193.27:0/557174559 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff8b0159a50 msgr2=0x7ff8b0163900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:25.920 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.913+0000 7ff8b750d6c0 1 --2- 10.20.193.27:0/557174559 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff8b0159a50 0x7ff8b0163900 secure :-1 s=READY pgs=254 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff8a40092e0 tx=0x7ff8a400bc50 comp rx=0 tx=0).stop 2026-02-13T02:22:25.920 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.913+0000 7ff8b750d6c0 1 -- 10.20.193.27:0/557174559 shutdown_connections 2026-02-13T02:22:25.920 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.913+0000 7ff8b750d6c0 1 --2- 10.20.193.27:0/557174559 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff88c0b5710 0x7ff88c0d5bc0 unknown :-1 s=CLOSED pgs=182 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:25.921 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.913+0000 7ff8b750d6c0 1 --2- 10.20.193.27:0/557174559 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff8b0168cc0 0x7ff8b0169160 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:25.921 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.913+0000 7ff8b750d6c0 1 --2- 10.20.193.27:0/557174559 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff8b0159a50 0x7ff8b0163900 unknown :-1 s=CLOSED pgs=254 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:25.921 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.913+0000 7ff8b750d6c0 1 --2- 10.20.193.27:0/557174559 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff8b0062fd0 0x7ff8b01633c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:25.921 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.913+0000 7ff8b750d6c0 1 -- 10.20.193.27:0/557174559 >> 10.20.193.27:0/557174559 conn(0x7ff8b00ecb00 msgr2=0x7ff8b010cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:25.921 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.913+0000 7ff8b750d6c0 1 -- 10.20.193.27:0/557174559 shutdown_connections 2026-02-13T02:22:25.921 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:25.913+0000 7ff8b750d6c0 1 -- 10.20.193.27:0/557174559 wait complete. 2026-02-13T02:22:25.955 INFO:teuthology.orchestra.run.trial027.stdout:{"epoch":12,"flags":0,"active_gid":14116,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6800","nonce":1837452956},{"type":"v1","addr":"10.20.193.27:6801","nonce":1837452956}]},"active_addr":"10.20.193.27:6801/1837452956","active_change":"2026-02-13T02:19:56.309520+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[{"gid":14337,"name":"z","mgr_features":4544132024016699391,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":14358,"name":"y","mgr_features":4544132024016699391,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":2,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.27:0","nonce":3818537008}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.27:0","nonce":2088025267}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.27:0","nonce":2269665231}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.27:0","nonce":2002591703}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.27:0","nonce":2249734914}]}]} 2026-02-13T02:22:25.983 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2026-02-13T02:22:25.983 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2026-02-13T02:22:25.983 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd dump --format=json 2026-02-13T02:22:26.111 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:26.226 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.222+0000 7fed634d06c0 1 -- 10.20.193.27:0/2805249843 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fed5c10f200 msgr2=0x7fed5c10f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:26.226 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.222+0000 7fed634d06c0 1 --2- 10.20.193.27:0/2805249843 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fed5c10f200 0x7fed5c10f5d0 secure :-1 s=READY pgs=255 gs=1 cs=0 l=1 c_cookie=2064139141321953643 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fed4c0099a0 tx=0x7fed4c02ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:26.226 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.222+0000 7fed634d06c0 1 -- 10.20.193.27:0/2805249843 shutdown_connections 2026-02-13T02:22:26.226 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.222+0000 7fed634d06c0 1 --2- 10.20.193.27:0/2805249843 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fed5c115410 0x7fed5c175db0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:26.226 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.222+0000 7fed634d06c0 1 --2- 10.20.193.27:0/2805249843 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fed5c10fbf0 0x7fed5c114c30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:26.226 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.222+0000 7fed634d06c0 1 --2- 10.20.193.27:0/2805249843 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fed5c10f200 0x7fed5c10f5d0 unknown :-1 s=CLOSED pgs=255 gs=1 cs=0 l=1 c_cookie=2064139141321953643 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:26.226 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.222+0000 7fed634d06c0 1 -- 10.20.193.27:0/2805249843 >> 10.20.193.27:0/2805249843 conn(0x7fed5c0ecb00 msgr2=0x7fed5c10cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:26.226 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.222+0000 7fed634d06c0 1 -- 10.20.193.27:0/2805249843 shutdown_connections 2026-02-13T02:22:26.227 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.222+0000 7fed634d06c0 1 -- 10.20.193.27:0/2805249843 wait complete. 2026-02-13T02:22:26.227 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.223+0000 7fed634d06c0 1 Processor -- start 2026-02-13T02:22:26.227 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.223+0000 7fed634d06c0 1 -- start start 2026-02-13T02:22:26.227 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.223+0000 7fed634d06c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fed5c10f200 0x7fed5c11d350 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:26.227 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.223+0000 7fed634d06c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fed5c115410 0x7fed5c11d890 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:26.227 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.223+0000 7fed634d06c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fed5c122c50 0x7fed5c1230f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:26.227 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.223+0000 7fed634d06c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c123660 con 0x7fed5c115410 2026-02-13T02:22:26.227 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.223+0000 7fed634d06c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c1237d0 con 0x7fed5c10f200 2026-02-13T02:22:26.227 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.223+0000 7fed634d06c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c123990 con 0x7fed5c122c50 2026-02-13T02:22:26.227 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.223+0000 7fed60a446c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fed5c115410 0x7fed5c11d890 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:26.227 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.223+0000 7fed60a446c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fed5c115410 0x7fed5c11d890 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55506/0 (socket says 10.20.193.27:55506) 2026-02-13T02:22:26.228 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.223+0000 7fed60a446c0 1 -- 10.20.193.27:0/3678262786 learned_addr learned my addr 10.20.193.27:0/3678262786 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:26.228 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.223+0000 7fed612456c0 1 --2- 10.20.193.27:0/3678262786 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fed5c10f200 0x7fed5c11d350 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:26.228 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.223+0000 7fed61a466c0 1 --2- 10.20.193.27:0/3678262786 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fed5c122c50 0x7fed5c1230f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:26.228 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.223+0000 7fed60a446c0 1 -- 10.20.193.27:0/3678262786 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fed5c122c50 msgr2=0x7fed5c1230f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:26.228 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.223+0000 7fed60a446c0 1 --2- 10.20.193.27:0/3678262786 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fed5c122c50 0x7fed5c1230f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:26.228 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.223+0000 7fed60a446c0 1 -- 10.20.193.27:0/3678262786 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fed5c10f200 msgr2=0x7fed5c11d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:26.228 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.223+0000 7fed60a446c0 1 --2- 10.20.193.27:0/3678262786 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fed5c10f200 0x7fed5c11d350 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:26.228 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.223+0000 7fed60a446c0 1 -- 10.20.193.27:0/3678262786 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed4c009590 con 0x7fed5c115410 2026-02-13T02:22:26.228 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.223+0000 7fed60a446c0 1 --2- 10.20.193.27:0/3678262786 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fed5c115410 0x7fed5c11d890 secure :-1 s=READY pgs=256 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fed5000b800 tx=0x7fed5000bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:26.228 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.223+0000 7fed4a7fc6c0 1 -- 10.20.193.27:0/3678262786 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fed50004a80 con 0x7fed5c115410 2026-02-13T02:22:26.228 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.223+0000 7fed4a7fc6c0 1 -- 10.20.193.27:0/3678262786 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fed50009070 con 0x7fed5c115410 2026-02-13T02:22:26.228 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.223+0000 7fed634d06c0 1 -- 10.20.193.27:0/3678262786 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed5c163e40 con 0x7fed5c115410 2026-02-13T02:22:26.229 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.223+0000 7fed4a7fc6c0 1 -- 10.20.193.27:0/3678262786 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fed5000cdd0 con 0x7fed5c115410 2026-02-13T02:22:26.229 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.223+0000 7fed634d06c0 1 -- 10.20.193.27:0/3678262786 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed5c164320 con 0x7fed5c115410 2026-02-13T02:22:26.229 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.224+0000 7fed634d06c0 1 -- 10.20.193.27:0/3678262786 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed5c1104d0 con 0x7fed5c115410 2026-02-13T02:22:26.230 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.225+0000 7fed4a7fc6c0 1 -- 10.20.193.27:0/3678262786 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7fed50004060 con 0x7fed5c115410 2026-02-13T02:22:26.230 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.226+0000 7fed4a7fc6c0 1 --2- 10.20.193.27:0/3678262786 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fed380b5710 0x7fed380d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:26.230 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.226+0000 7fed612456c0 1 --2- 10.20.193.27:0/3678262786 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fed380b5710 0x7fed380d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:26.230 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.226+0000 7fed4a7fc6c0 1 -- 10.20.193.27:0/3678262786 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7fed500e3920 con 0x7fed5c115410 2026-02-13T02:22:26.230 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.226+0000 7fed612456c0 1 --2- 10.20.193.27:0/3678262786 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fed380b5710 0x7fed380d5bc0 secure :-1 s=READY pgs=183 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fed4c005b40 tx=0x7fed4c0023a0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:26.230 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.226+0000 7fed4a7fc6c0 1 -- 10.20.193.27:0/3678262786 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fed500a9de0 con 0x7fed5c115410 2026-02-13T02:22:26.320 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.319+0000 7fed634d06c0 1 -- 10.20.193.27:0/3678262786 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed5c10fbf0 con 0x7fed5c115410 2026-02-13T02:22:26.320 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:26 trial027 ceph-mon[24567]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-13T02:22:26.321 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:26 trial027 ceph-mon[24567]: pgmap v188: 97 pgs: 97 active+clean; 586 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2026-02-13T02:22:26.321 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:26 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3655885843' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2026-02-13T02:22:26.321 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:26 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/557174559' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2026-02-13T02:22:26.321 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.320+0000 7fed4a7fc6c0 1 -- 10.20.193.27:0/3678262786 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+22002 (secure 0 0 0) 0x7fed500a9530 con 0x7fed5c115410 2026-02-13T02:22:26.322 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:22:26.322 INFO:teuthology.orchestra.run.trial027.stdout:{"epoch":70,"fsid":"707dbb29-0882-11f1-9262-d404e6e7d460","created":"2026-02-13T02:19:50.420575+0000","modified":"2026-02-13T02:22:18.009553+0000","last_up_change":"2026-02-13T02:21:58.960121+0000","last_in_change":"2026-02-13T02:21:50.695981+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-13T02:20:54.702258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2026-02-13T02:22:14.280021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2026-02-13T02:22:15.383271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1a7af29f-2999-4be3-a38e-3fc7feb9f2f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":7,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6802","nonce":832083434},{"type":"v1","addr":"10.20.193.27:6803","nonce":832083434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6804","nonce":832083434},{"type":"v1","addr":"10.20.193.27:6805","nonce":832083434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6808","nonce":832083434},{"type":"v1","addr":"10.20.193.27:6809","nonce":832083434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6806","nonce":832083434},{"type":"v1","addr":"10.20.193.27:6807","nonce":832083434}]},"public_addr":"10.20.193.27:6803/832083434","cluster_addr":"10.20.193.27:6805/832083434","heartbeat_back_addr":"10.20.193.27:6809/832083434","heartbeat_front_addr":"10.20.193.27:6807/832083434","state":["exists","up"]},{"osd":1,"uuid":"8c0ad8cf-4150-43f6-8a9c-258b0ebf6a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6800","nonce":4070516162},{"type":"v1","addr":"10.20.193.28:6801","nonce":4070516162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6802","nonce":4070516162},{"type":"v1","addr":"10.20.193.28:6803","nonce":4070516162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6806","nonce":4070516162},{"type":"v1","addr":"10.20.193.28:6807","nonce":4070516162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6804","nonce":4070516162},{"type":"v1","addr":"10.20.193.28:6805","nonce":4070516162}]},"public_addr":"10.20.193.28:6801/4070516162","cluster_addr":"10.20.193.28:6803/4070516162","heartbeat_back_addr":"10.20.193.28:6807/4070516162","heartbeat_front_addr":"10.20.193.28:6805/4070516162","state":["exists","up"]},{"osd":2,"uuid":"a7813078-94d6-4559-a89a-5525161ab5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6800","nonce":4075625103},{"type":"v1","addr":"10.20.193.107:6801","nonce":4075625103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6802","nonce":4075625103},{"type":"v1","addr":"10.20.193.107:6803","nonce":4075625103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6806","nonce":4075625103},{"type":"v1","addr":"10.20.193.107:6807","nonce":4075625103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6804","nonce":4075625103},{"type":"v1","addr":"10.20.193.107:6805","nonce":4075625103}]},"public_addr":"10.20.193.107:6801/4075625103","cluster_addr":"10.20.193.107:6803/4075625103","heartbeat_back_addr":"10.20.193.107:6807/4075625103","heartbeat_front_addr":"10.20.193.107:6805/4075625103","state":["exists","up"]},{"osd":3,"uuid":"268f9714-faed-452b-ae34-3240fa23153a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6810","nonce":382071678},{"type":"v1","addr":"10.20.193.27:6811","nonce":382071678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6812","nonce":382071678},{"type":"v1","addr":"10.20.193.27:6813","nonce":382071678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6816","nonce":382071678},{"type":"v1","addr":"10.20.193.27:6817","nonce":382071678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6814","nonce":382071678},{"type":"v1","addr":"10.20.193.27:6815","nonce":382071678}]},"public_addr":"10.20.193.27:6811/382071678","cluster_addr":"10.20.193.27:6813/382071678","heartbeat_back_addr":"10.20.193.27:6817/382071678","heartbeat_front_addr":"10.20.193.27:6815/382071678","state":["exists","up"]},{"osd":4,"uuid":"4beaec0d-7b39-4bb0-9770-3dc4c848b106","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6808","nonce":711804032},{"type":"v1","addr":"10.20.193.28:6809","nonce":711804032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6810","nonce":711804032},{"type":"v1","addr":"10.20.193.28:6811","nonce":711804032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6814","nonce":711804032},{"type":"v1","addr":"10.20.193.28:6815","nonce":711804032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6812","nonce":711804032},{"type":"v1","addr":"10.20.193.28:6813","nonce":711804032}]},"public_addr":"10.20.193.28:6809/711804032","cluster_addr":"10.20.193.28:6811/711804032","heartbeat_back_addr":"10.20.193.28:6815/711804032","heartbeat_front_addr":"10.20.193.28:6813/711804032","state":["exists","up"]},{"osd":5,"uuid":"bb16754e-66f2-4951-9f31-d1dde591ec92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6808","nonce":4246100128},{"type":"v1","addr":"10.20.193.107:6809","nonce":4246100128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6810","nonce":4246100128},{"type":"v1","addr":"10.20.193.107:6811","nonce":4246100128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6814","nonce":4246100128},{"type":"v1","addr":"10.20.193.107:6815","nonce":4246100128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6812","nonce":4246100128},{"type":"v1","addr":"10.20.193.107:6813","nonce":4246100128}]},"public_addr":"10.20.193.107:6809/4246100128","cluster_addr":"10.20.193.107:6811/4246100128","heartbeat_back_addr":"10.20.193.107:6815/4246100128","heartbeat_front_addr":"10.20.193.107:6813/4246100128","state":["exists","up"]},{"osd":6,"uuid":"318afff7-b7c9-4b19-a7ba-06e7ab122ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6818","nonce":3514139178},{"type":"v1","addr":"10.20.193.27:6819","nonce":3514139178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6820","nonce":3514139178},{"type":"v1","addr":"10.20.193.27:6821","nonce":3514139178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6824","nonce":3514139178},{"type":"v1","addr":"10.20.193.27:6825","nonce":3514139178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6822","nonce":3514139178},{"type":"v1","addr":"10.20.193.27:6823","nonce":3514139178}]},"public_addr":"10.20.193.27:6819/3514139178","cluster_addr":"10.20.193.27:6821/3514139178","heartbeat_back_addr":"10.20.193.27:6825/3514139178","heartbeat_front_addr":"10.20.193.27:6823/3514139178","state":["exists","up"]},{"osd":7,"uuid":"b545439a-3f32-4b0a-bc6a-17f0d8050c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6816","nonce":1405200379},{"type":"v1","addr":"10.20.193.28:6817","nonce":1405200379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6818","nonce":1405200379},{"type":"v1","addr":"10.20.193.28:6819","nonce":1405200379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6822","nonce":1405200379},{"type":"v1","addr":"10.20.193.28:6823","nonce":1405200379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6820","nonce":1405200379},{"type":"v1","addr":"10.20.193.28:6821","nonce":1405200379}]},"public_addr":"10.20.193.28:6817/1405200379","cluster_addr":"10.20.193.28:6819/1405200379","heartbeat_back_addr":"10.20.193.28:6823/1405200379","heartbeat_front_addr":"10.20.193.28:6821/1405200379","state":["exists","up"]},{"osd":8,"uuid":"a7c6bbba-760c-484a-8171-44c70f7d9c89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6816","nonce":1331562104},{"type":"v1","addr":"10.20.193.107:6817","nonce":1331562104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6818","nonce":1331562104},{"type":"v1","addr":"10.20.193.107:6819","nonce":1331562104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6822","nonce":1331562104},{"type":"v1","addr":"10.20.193.107:6823","nonce":1331562104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6820","nonce":1331562104},{"type":"v1","addr":"10.20.193.107:6821","nonce":1331562104}]},"public_addr":"10.20.193.107:6817/1331562104","cluster_addr":"10.20.193.107:6819/1331562104","heartbeat_back_addr":"10.20.193.107:6823/1331562104","heartbeat_front_addr":"10.20.193.107:6821/1331562104","state":["exists","up"]},{"osd":9,"uuid":"72cbd422-3c3a-4904-9075-f543671d4f48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6826","nonce":1444927853},{"type":"v1","addr":"10.20.193.27:6827","nonce":1444927853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6828","nonce":1444927853},{"type":"v1","addr":"10.20.193.27:6829","nonce":1444927853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6832","nonce":1444927853},{"type":"v1","addr":"10.20.193.27:6833","nonce":1444927853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6830","nonce":1444927853},{"type":"v1","addr":"10.20.193.27:6831","nonce":1444927853}]},"public_addr":"10.20.193.27:6827/1444927853","cluster_addr":"10.20.193.27:6829/1444927853","heartbeat_back_addr":"10.20.193.27:6833/1444927853","heartbeat_front_addr":"10.20.193.27:6831/1444927853","state":["exists","up"]},{"osd":10,"uuid":"4bb7b75e-adf7-4c38-9343-dba513b57ed7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6824","nonce":2255755797},{"type":"v1","addr":"10.20.193.28:6825","nonce":2255755797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6826","nonce":2255755797},{"type":"v1","addr":"10.20.193.28:6827","nonce":2255755797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6830","nonce":2255755797},{"type":"v1","addr":"10.20.193.28:6831","nonce":2255755797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6828","nonce":2255755797},{"type":"v1","addr":"10.20.193.28:6829","nonce":2255755797}]},"public_addr":"10.20.193.28:6825/2255755797","cluster_addr":"10.20.193.28:6827/2255755797","heartbeat_back_addr":"10.20.193.28:6831/2255755797","heartbeat_front_addr":"10.20.193.28:6829/2255755797","state":["exists","up"]},{"osd":11,"uuid":"115fe8c6-77fc-41dd-ade0-db8eea024974","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6824","nonce":1496433214},{"type":"v1","addr":"10.20.193.107:6825","nonce":1496433214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6826","nonce":1496433214},{"type":"v1","addr":"10.20.193.107:6827","nonce":1496433214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6830","nonce":1496433214},{"type":"v1","addr":"10.20.193.107:6831","nonce":1496433214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6828","nonce":1496433214},{"type":"v1","addr":"10.20.193.107:6829","nonce":1496433214}]},"public_addr":"10.20.193.107:6825/1496433214","cluster_addr":"10.20.193.107:6827/1496433214","heartbeat_back_addr":"10.20.193.107:6831/1496433214","heartbeat_front_addr":"10.20.193.107:6829/1496433214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:20:37.596432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:20:44.867765+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:20:51.765273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:20:58.994342+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:06.232104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:13.195531+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:20.453172+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:27.721147+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:35.051979+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:42.463998+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:49.889642+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:57.167778+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"10.20.193.27:0/1486586326":"2026-02-14T02:19:56.309429+0000","10.20.193.27:6801/3576730225":"2026-02-14T02:19:56.309429+0000","10.20.193.27:6800/3576730225":"2026-02-14T02:19:56.309429+0000","10.20.193.27:0/2609227888":"2026-02-14T02:19:56.309429+0000","10.20.193.27:0/1389740324":"2026-02-14T02:19:56.309429+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-13T02:22:26.323 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.321+0000 7fed634d06c0 1 -- 10.20.193.27:0/3678262786 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fed380b5710 msgr2=0x7fed380d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:26.323 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.321+0000 7fed634d06c0 1 --2- 10.20.193.27:0/3678262786 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fed380b5710 0x7fed380d5bc0 secure :-1 s=READY pgs=183 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fed4c005b40 tx=0x7fed4c0023a0 comp rx=0 tx=0).stop 2026-02-13T02:22:26.323 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.321+0000 7fed634d06c0 1 -- 10.20.193.27:0/3678262786 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fed5c115410 msgr2=0x7fed5c11d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:26.323 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.321+0000 7fed634d06c0 1 --2- 10.20.193.27:0/3678262786 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fed5c115410 0x7fed5c11d890 secure :-1 s=READY pgs=256 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fed5000b800 tx=0x7fed5000bcd0 comp rx=0 tx=0).stop 2026-02-13T02:22:26.323 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.321+0000 7fed634d06c0 1 -- 10.20.193.27:0/3678262786 shutdown_connections 2026-02-13T02:22:26.324 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.321+0000 7fed634d06c0 1 --2- 10.20.193.27:0/3678262786 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fed380b5710 0x7fed380d5bc0 unknown :-1 s=CLOSED pgs=183 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:26.324 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.321+0000 7fed634d06c0 1 --2- 10.20.193.27:0/3678262786 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fed5c122c50 0x7fed5c1230f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:26.324 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.321+0000 7fed634d06c0 1 --2- 10.20.193.27:0/3678262786 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fed5c115410 0x7fed5c11d890 unknown :-1 s=CLOSED pgs=256 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:26.324 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.321+0000 7fed634d06c0 1 --2- 10.20.193.27:0/3678262786 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fed5c10f200 0x7fed5c11d350 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:26.324 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.321+0000 7fed634d06c0 1 -- 10.20.193.27:0/3678262786 >> 10.20.193.27:0/3678262786 conn(0x7fed5c0ecb00 msgr2=0x7fed5c10cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:26.324 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.321+0000 7fed634d06c0 1 -- 10.20.193.27:0/3678262786 shutdown_connections 2026-02-13T02:22:26.324 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.321+0000 7fed634d06c0 1 -- 10.20.193.27:0/3678262786 wait complete. 2026-02-13T02:22:26.363 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2026-02-13T02:22:26.364 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd dump --format=json 2026-02-13T02:22:26.480 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:26.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:26 trial028 ceph-mon[26701]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-13T02:22:26.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:26 trial028 ceph-mon[26701]: pgmap v188: 97 pgs: 97 active+clean; 586 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2026-02-13T02:22:26.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:26 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3655885843' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2026-02-13T02:22:26.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:26 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/557174559' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2026-02-13T02:22:26.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:26 trial107 ceph-mon[27422]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-13T02:22:26.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:26 trial107 ceph-mon[27422]: pgmap v188: 97 pgs: 97 active+clean; 586 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2026-02-13T02:22:26.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:26 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3655885843' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2026-02-13T02:22:26.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:26 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/557174559' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2026-02-13T02:22:26.597 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.594+0000 7fedd54a76c0 1 -- 10.20.193.27:0/2315669341 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fedd01556e0 msgr2=0x7fedd0175ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:26.597 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.594+0000 7fedd54a76c0 1 --2- 10.20.193.27:0/2315669341 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fedd01556e0 0x7fedd0175ac0 secure :-1 s=READY pgs=257 gs=1 cs=0 l=1 c_cookie=8138411108446124429 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fedc000aa40 tx=0x7fedc002fdf0 comp rx=0 tx=0).stop 2026-02-13T02:22:26.597 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.594+0000 7fedd54a76c0 1 -- 10.20.193.27:0/2315669341 shutdown_connections 2026-02-13T02:22:26.597 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.594+0000 7fedd54a76c0 1 --2- 10.20.193.27:0/2315669341 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fedd01556e0 0x7fedd0175ac0 unknown :-1 s=CLOSED pgs=257 gs=1 cs=0 l=1 c_cookie=8138411108446124429 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:26.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.594+0000 7fedd54a76c0 1 --2- 10.20.193.27:0/2315669341 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fedd0061030 0x7fedd01551a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:26.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.594+0000 7fedd54a76c0 1 --2- 10.20.193.27:0/2315669341 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fedd0060720 0x7fedd0060af0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:26.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.594+0000 7fedd54a76c0 1 -- 10.20.193.27:0/2315669341 >> 10.20.193.27:0/2315669341 conn(0x7fedd00ea8b0 msgr2=0x7fedd010acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:26.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.594+0000 7fedd54a76c0 1 -- 10.20.193.27:0/2315669341 shutdown_connections 2026-02-13T02:22:26.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.594+0000 7fedd54a76c0 1 -- 10.20.193.27:0/2315669341 wait complete. 2026-02-13T02:22:26.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.594+0000 7fedd54a76c0 1 Processor -- start 2026-02-13T02:22:26.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.594+0000 7fedd54a76c0 1 -- start start 2026-02-13T02:22:26.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.594+0000 7fedd54a76c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fedd0060720 0x7fedd01614e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:26.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.594+0000 7fedd54a76c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fedd0061030 0x7fedd0161a20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:26.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.594+0000 7fedd54a76c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fedd01556e0 0x7fedd0166de0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:26.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.594+0000 7fedd54a76c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fedd01673a0 con 0x7fedd0061030 2026-02-13T02:22:26.599 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.594+0000 7fedd54a76c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fedd0167510 con 0x7fedd0060720 2026-02-13T02:22:26.599 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.594+0000 7fedd54a76c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fedd01676d0 con 0x7fedd01556e0 2026-02-13T02:22:26.599 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.594+0000 7fedce7fc6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fedd0061030 0x7fedd0161a20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:26.599 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.594+0000 7fedce7fc6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fedd0061030 0x7fedd0161a20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55528/0 (socket says 10.20.193.27:55528) 2026-02-13T02:22:26.599 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.594+0000 7fedce7fc6c0 1 -- 10.20.193.27:0/3132368379 learned_addr learned my addr 10.20.193.27:0/3132368379 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:26.599 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.594+0000 7fedcf7fe6c0 1 --2- 10.20.193.27:0/3132368379 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fedd01556e0 0x7fedd0166de0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:26.599 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.595+0000 7fedce7fc6c0 1 -- 10.20.193.27:0/3132368379 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fedd01556e0 msgr2=0x7fedd0166de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:26.599 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.595+0000 7fedce7fc6c0 1 --2- 10.20.193.27:0/3132368379 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fedd01556e0 0x7fedd0166de0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:26.599 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.595+0000 7fedce7fc6c0 1 -- 10.20.193.27:0/3132368379 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fedd0060720 msgr2=0x7fedd01614e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:26.599 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.595+0000 7fedce7fc6c0 1 --2- 10.20.193.27:0/3132368379 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fedd0060720 0x7fedd01614e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:26.599 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.595+0000 7fedce7fc6c0 1 -- 10.20.193.27:0/3132368379 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc000a660 con 0x7fedd0061030 2026-02-13T02:22:26.599 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.595+0000 7fedce7fc6c0 1 --2- 10.20.193.27:0/3132368379 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fedd0061030 0x7fedd0161a20 secure :-1 s=READY pgs=258 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fedb800b6c0 tx=0x7fedb800bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:26.600 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.595+0000 7fedaffff6c0 1 -- 10.20.193.27:0/3132368379 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fedb8004560 con 0x7fedd0061030 2026-02-13T02:22:26.600 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.595+0000 7fedaffff6c0 1 -- 10.20.193.27:0/3132368379 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fedb8009030 con 0x7fedd0061030 2026-02-13T02:22:26.600 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.595+0000 7fedaffff6c0 1 -- 10.20.193.27:0/3132368379 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fedb800cc80 con 0x7fedd0061030 2026-02-13T02:22:26.600 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.595+0000 7fedd54a76c0 1 -- 10.20.193.27:0/3132368379 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd01679b0 con 0x7fedd0061030 2026-02-13T02:22:26.600 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.595+0000 7fedd54a76c0 1 -- 10.20.193.27:0/3132368379 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd0151310 con 0x7fedd0061030 2026-02-13T02:22:26.600 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.595+0000 7fedd54a76c0 1 -- 10.20.193.27:0/3132368379 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedd0061c30 con 0x7fedd0061030 2026-02-13T02:22:26.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.597+0000 7fedaffff6c0 1 -- 10.20.193.27:0/3132368379 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7fedb8005080 con 0x7fedd0061030 2026-02-13T02:22:26.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.597+0000 7fedaffff6c0 1 --2- 10.20.193.27:0/3132368379 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7feda40b5690 0x7feda40d5b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:26.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.597+0000 7fedaffff6c0 1 -- 10.20.193.27:0/3132368379 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7fedb80e39f0 con 0x7fedd0061030 2026-02-13T02:22:26.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.597+0000 7fedceffd6c0 1 --2- 10.20.193.27:0/3132368379 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7feda40b5690 0x7feda40d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:26.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.597+0000 7fedceffd6c0 1 --2- 10.20.193.27:0/3132368379 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7feda40b5690 0x7feda40d5b40 secure :-1 s=READY pgs=184 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fedc4005980 tx=0x7fedc4005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:26.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.598+0000 7fedaffff6c0 1 -- 10.20.193.27:0/3132368379 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fedb80a6450 con 0x7fedd0061030 2026-02-13T02:22:26.690 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.688+0000 7fedd54a76c0 1 -- 10.20.193.27:0/3132368379 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fedd0167ba0 con 0x7fedd0061030 2026-02-13T02:22:26.690 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.689+0000 7fedaffff6c0 1 -- 10.20.193.27:0/3132368379 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+22002 (secure 0 0 0) 0x7fedb80a9600 con 0x7fedd0061030 2026-02-13T02:22:26.691 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:22:26.691 INFO:teuthology.orchestra.run.trial027.stdout:{"epoch":70,"fsid":"707dbb29-0882-11f1-9262-d404e6e7d460","created":"2026-02-13T02:19:50.420575+0000","modified":"2026-02-13T02:22:18.009553+0000","last_up_change":"2026-02-13T02:21:58.960121+0000","last_in_change":"2026-02-13T02:21:50.695981+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-13T02:20:54.702258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2026-02-13T02:22:14.280021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2026-02-13T02:22:15.383271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1a7af29f-2999-4be3-a38e-3fc7feb9f2f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":7,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6802","nonce":832083434},{"type":"v1","addr":"10.20.193.27:6803","nonce":832083434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6804","nonce":832083434},{"type":"v1","addr":"10.20.193.27:6805","nonce":832083434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6808","nonce":832083434},{"type":"v1","addr":"10.20.193.27:6809","nonce":832083434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6806","nonce":832083434},{"type":"v1","addr":"10.20.193.27:6807","nonce":832083434}]},"public_addr":"10.20.193.27:6803/832083434","cluster_addr":"10.20.193.27:6805/832083434","heartbeat_back_addr":"10.20.193.27:6809/832083434","heartbeat_front_addr":"10.20.193.27:6807/832083434","state":["exists","up"]},{"osd":1,"uuid":"8c0ad8cf-4150-43f6-8a9c-258b0ebf6a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6800","nonce":4070516162},{"type":"v1","addr":"10.20.193.28:6801","nonce":4070516162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6802","nonce":4070516162},{"type":"v1","addr":"10.20.193.28:6803","nonce":4070516162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6806","nonce":4070516162},{"type":"v1","addr":"10.20.193.28:6807","nonce":4070516162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6804","nonce":4070516162},{"type":"v1","addr":"10.20.193.28:6805","nonce":4070516162}]},"public_addr":"10.20.193.28:6801/4070516162","cluster_addr":"10.20.193.28:6803/4070516162","heartbeat_back_addr":"10.20.193.28:6807/4070516162","heartbeat_front_addr":"10.20.193.28:6805/4070516162","state":["exists","up"]},{"osd":2,"uuid":"a7813078-94d6-4559-a89a-5525161ab5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6800","nonce":4075625103},{"type":"v1","addr":"10.20.193.107:6801","nonce":4075625103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6802","nonce":4075625103},{"type":"v1","addr":"10.20.193.107:6803","nonce":4075625103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6806","nonce":4075625103},{"type":"v1","addr":"10.20.193.107:6807","nonce":4075625103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6804","nonce":4075625103},{"type":"v1","addr":"10.20.193.107:6805","nonce":4075625103}]},"public_addr":"10.20.193.107:6801/4075625103","cluster_addr":"10.20.193.107:6803/4075625103","heartbeat_back_addr":"10.20.193.107:6807/4075625103","heartbeat_front_addr":"10.20.193.107:6805/4075625103","state":["exists","up"]},{"osd":3,"uuid":"268f9714-faed-452b-ae34-3240fa23153a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6810","nonce":382071678},{"type":"v1","addr":"10.20.193.27:6811","nonce":382071678}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6812","nonce":382071678},{"type":"v1","addr":"10.20.193.27:6813","nonce":382071678}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6816","nonce":382071678},{"type":"v1","addr":"10.20.193.27:6817","nonce":382071678}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6814","nonce":382071678},{"type":"v1","addr":"10.20.193.27:6815","nonce":382071678}]},"public_addr":"10.20.193.27:6811/382071678","cluster_addr":"10.20.193.27:6813/382071678","heartbeat_back_addr":"10.20.193.27:6817/382071678","heartbeat_front_addr":"10.20.193.27:6815/382071678","state":["exists","up"]},{"osd":4,"uuid":"4beaec0d-7b39-4bb0-9770-3dc4c848b106","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6808","nonce":711804032},{"type":"v1","addr":"10.20.193.28:6809","nonce":711804032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6810","nonce":711804032},{"type":"v1","addr":"10.20.193.28:6811","nonce":711804032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6814","nonce":711804032},{"type":"v1","addr":"10.20.193.28:6815","nonce":711804032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6812","nonce":711804032},{"type":"v1","addr":"10.20.193.28:6813","nonce":711804032}]},"public_addr":"10.20.193.28:6809/711804032","cluster_addr":"10.20.193.28:6811/711804032","heartbeat_back_addr":"10.20.193.28:6815/711804032","heartbeat_front_addr":"10.20.193.28:6813/711804032","state":["exists","up"]},{"osd":5,"uuid":"bb16754e-66f2-4951-9f31-d1dde591ec92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6808","nonce":4246100128},{"type":"v1","addr":"10.20.193.107:6809","nonce":4246100128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6810","nonce":4246100128},{"type":"v1","addr":"10.20.193.107:6811","nonce":4246100128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6814","nonce":4246100128},{"type":"v1","addr":"10.20.193.107:6815","nonce":4246100128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6812","nonce":4246100128},{"type":"v1","addr":"10.20.193.107:6813","nonce":4246100128}]},"public_addr":"10.20.193.107:6809/4246100128","cluster_addr":"10.20.193.107:6811/4246100128","heartbeat_back_addr":"10.20.193.107:6815/4246100128","heartbeat_front_addr":"10.20.193.107:6813/4246100128","state":["exists","up"]},{"osd":6,"uuid":"318afff7-b7c9-4b19-a7ba-06e7ab122ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6818","nonce":3514139178},{"type":"v1","addr":"10.20.193.27:6819","nonce":3514139178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6820","nonce":3514139178},{"type":"v1","addr":"10.20.193.27:6821","nonce":3514139178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6824","nonce":3514139178},{"type":"v1","addr":"10.20.193.27:6825","nonce":3514139178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6822","nonce":3514139178},{"type":"v1","addr":"10.20.193.27:6823","nonce":3514139178}]},"public_addr":"10.20.193.27:6819/3514139178","cluster_addr":"10.20.193.27:6821/3514139178","heartbeat_back_addr":"10.20.193.27:6825/3514139178","heartbeat_front_addr":"10.20.193.27:6823/3514139178","state":["exists","up"]},{"osd":7,"uuid":"b545439a-3f32-4b0a-bc6a-17f0d8050c07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6816","nonce":1405200379},{"type":"v1","addr":"10.20.193.28:6817","nonce":1405200379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6818","nonce":1405200379},{"type":"v1","addr":"10.20.193.28:6819","nonce":1405200379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6822","nonce":1405200379},{"type":"v1","addr":"10.20.193.28:6823","nonce":1405200379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6820","nonce":1405200379},{"type":"v1","addr":"10.20.193.28:6821","nonce":1405200379}]},"public_addr":"10.20.193.28:6817/1405200379","cluster_addr":"10.20.193.28:6819/1405200379","heartbeat_back_addr":"10.20.193.28:6823/1405200379","heartbeat_front_addr":"10.20.193.28:6821/1405200379","state":["exists","up"]},{"osd":8,"uuid":"a7c6bbba-760c-484a-8171-44c70f7d9c89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6816","nonce":1331562104},{"type":"v1","addr":"10.20.193.107:6817","nonce":1331562104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6818","nonce":1331562104},{"type":"v1","addr":"10.20.193.107:6819","nonce":1331562104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6822","nonce":1331562104},{"type":"v1","addr":"10.20.193.107:6823","nonce":1331562104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6820","nonce":1331562104},{"type":"v1","addr":"10.20.193.107:6821","nonce":1331562104}]},"public_addr":"10.20.193.107:6817/1331562104","cluster_addr":"10.20.193.107:6819/1331562104","heartbeat_back_addr":"10.20.193.107:6823/1331562104","heartbeat_front_addr":"10.20.193.107:6821/1331562104","state":["exists","up"]},{"osd":9,"uuid":"72cbd422-3c3a-4904-9075-f543671d4f48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6826","nonce":1444927853},{"type":"v1","addr":"10.20.193.27:6827","nonce":1444927853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6828","nonce":1444927853},{"type":"v1","addr":"10.20.193.27:6829","nonce":1444927853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6832","nonce":1444927853},{"type":"v1","addr":"10.20.193.27:6833","nonce":1444927853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.27:6830","nonce":1444927853},{"type":"v1","addr":"10.20.193.27:6831","nonce":1444927853}]},"public_addr":"10.20.193.27:6827/1444927853","cluster_addr":"10.20.193.27:6829/1444927853","heartbeat_back_addr":"10.20.193.27:6833/1444927853","heartbeat_front_addr":"10.20.193.27:6831/1444927853","state":["exists","up"]},{"osd":10,"uuid":"4bb7b75e-adf7-4c38-9343-dba513b57ed7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6824","nonce":2255755797},{"type":"v1","addr":"10.20.193.28:6825","nonce":2255755797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6826","nonce":2255755797},{"type":"v1","addr":"10.20.193.28:6827","nonce":2255755797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6830","nonce":2255755797},{"type":"v1","addr":"10.20.193.28:6831","nonce":2255755797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6828","nonce":2255755797},{"type":"v1","addr":"10.20.193.28:6829","nonce":2255755797}]},"public_addr":"10.20.193.28:6825/2255755797","cluster_addr":"10.20.193.28:6827/2255755797","heartbeat_back_addr":"10.20.193.28:6831/2255755797","heartbeat_front_addr":"10.20.193.28:6829/2255755797","state":["exists","up"]},{"osd":11,"uuid":"115fe8c6-77fc-41dd-ade0-db8eea024974","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6824","nonce":1496433214},{"type":"v1","addr":"10.20.193.107:6825","nonce":1496433214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6826","nonce":1496433214},{"type":"v1","addr":"10.20.193.107:6827","nonce":1496433214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6830","nonce":1496433214},{"type":"v1","addr":"10.20.193.107:6831","nonce":1496433214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.107:6828","nonce":1496433214},{"type":"v1","addr":"10.20.193.107:6829","nonce":1496433214}]},"public_addr":"10.20.193.107:6825/1496433214","cluster_addr":"10.20.193.107:6827/1496433214","heartbeat_back_addr":"10.20.193.107:6831/1496433214","heartbeat_front_addr":"10.20.193.107:6829/1496433214","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:20:37.596432+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:20:44.867765+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:20:51.765273+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:20:58.994342+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:06.232104+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:13.195531+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:20.453172+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:27.721147+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:35.051979+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:42.463998+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:49.889642+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-13T02:21:57.167778+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"10.20.193.27:0/1486586326":"2026-02-14T02:19:56.309429+0000","10.20.193.27:6801/3576730225":"2026-02-14T02:19:56.309429+0000","10.20.193.27:6800/3576730225":"2026-02-14T02:19:56.309429+0000","10.20.193.27:0/2609227888":"2026-02-14T02:19:56.309429+0000","10.20.193.27:0/1389740324":"2026-02-14T02:19:56.309429+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-13T02:22:26.692 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.690+0000 7fedd54a76c0 1 -- 10.20.193.27:0/3132368379 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7feda40b5690 msgr2=0x7feda40d5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:26.692 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.690+0000 7fedd54a76c0 1 --2- 10.20.193.27:0/3132368379 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7feda40b5690 0x7feda40d5b40 secure :-1 s=READY pgs=184 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fedc4005980 tx=0x7fedc4005910 comp rx=0 tx=0).stop 2026-02-13T02:22:26.692 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.690+0000 7fedd54a76c0 1 -- 10.20.193.27:0/3132368379 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fedd0061030 msgr2=0x7fedd0161a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:26.692 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.690+0000 7fedd54a76c0 1 --2- 10.20.193.27:0/3132368379 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fedd0061030 0x7fedd0161a20 secure :-1 s=READY pgs=258 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fedb800b6c0 tx=0x7fedb800bb90 comp rx=0 tx=0).stop 2026-02-13T02:22:26.693 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.690+0000 7fedd54a76c0 1 -- 10.20.193.27:0/3132368379 shutdown_connections 2026-02-13T02:22:26.693 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.690+0000 7fedd54a76c0 1 --2- 10.20.193.27:0/3132368379 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7feda40b5690 0x7feda40d5b40 unknown :-1 s=CLOSED pgs=184 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:26.693 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.690+0000 7fedd54a76c0 1 --2- 10.20.193.27:0/3132368379 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fedd01556e0 0x7fedd0166de0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:26.693 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.690+0000 7fedd54a76c0 1 --2- 10.20.193.27:0/3132368379 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fedd0061030 0x7fedd0161a20 unknown :-1 s=CLOSED pgs=258 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:26.693 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.690+0000 7fedd54a76c0 1 --2- 10.20.193.27:0/3132368379 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fedd0060720 0x7fedd01614e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:26.693 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.690+0000 7fedd54a76c0 1 -- 10.20.193.27:0/3132368379 >> 10.20.193.27:0/3132368379 conn(0x7fedd00ea8b0 msgr2=0x7fedd010ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:26.693 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.690+0000 7fedd54a76c0 1 -- 10.20.193.27:0/3132368379 shutdown_connections 2026-02-13T02:22:26.693 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:26.690+0000 7fedd54a76c0 1 -- 10.20.193.27:0/3132368379 wait complete. 2026-02-13T02:22:26.735 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph tell osd.0 flush_pg_stats 2026-02-13T02:22:26.736 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph tell osd.1 flush_pg_stats 2026-02-13T02:22:26.736 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph tell osd.2 flush_pg_stats 2026-02-13T02:22:26.737 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph tell osd.3 flush_pg_stats 2026-02-13T02:22:26.737 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph tell osd.4 flush_pg_stats 2026-02-13T02:22:26.737 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph tell osd.5 flush_pg_stats 2026-02-13T02:22:26.738 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph tell osd.6 flush_pg_stats 2026-02-13T02:22:26.738 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph tell osd.7 flush_pg_stats 2026-02-13T02:22:26.739 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph tell osd.8 flush_pg_stats 2026-02-13T02:22:26.739 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph tell osd.9 flush_pg_stats 2026-02-13T02:22:26.740 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph tell osd.10 flush_pg_stats 2026-02-13T02:22:26.740 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph tell osd.11 flush_pg_stats 2026-02-13T02:22:26.871 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:26.874 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:26.874 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:26.875 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:26.875 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:26.879 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:26.880 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:26.881 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:26.884 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:26.886 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:26.887 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:26.891 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:27.008 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.007+0000 7f003533c6c0 1 -- 10.20.193.27:0/2584269091 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f003010fc20 msgr2=0x7f0030114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.007+0000 7f003533c6c0 1 --2- 10.20.193.27:0/2584269091 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f003010fc20 0x7f0030114c60 secure :-1 s=READY pgs=259 gs=2 cs=0 l=1 c_cookie=14613574553857598387 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f00180099a0 tx=0x7f001802ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:27.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.007+0000 7f003533c6c0 1 -- 10.20.193.27:0/2584269091 shutdown_connections 2026-02-13T02:22:27.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.007+0000 7f003533c6c0 1 --2- 10.20.193.27:0/2584269091 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0030115440 0x7f0030175de0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.007+0000 7f003533c6c0 1 --2- 10.20.193.27:0/2584269091 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f003010fc20 0x7f0030114c60 unknown :-1 s=CLOSED pgs=259 gs=2 cs=0 l=1 c_cookie=14613574553857598387 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.007+0000 7f003533c6c0 1 --2- 10.20.193.27:0/2584269091 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f003010f230 0x7f003010f600 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.007+0000 7f003533c6c0 1 -- 10.20.193.27:0/2584269091 >> 10.20.193.27:0/2584269091 conn(0x7f00300ecb30 msgr2=0x7f003010cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.007+0000 7f003533c6c0 1 -- 10.20.193.27:0/2584269091 shutdown_connections 2026-02-13T02:22:27.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.007+0000 7f003533c6c0 1 -- 10.20.193.27:0/2584269091 wait complete. 2026-02-13T02:22:27.010 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.008+0000 7f003533c6c0 1 Processor -- start 2026-02-13T02:22:27.010 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.008+0000 7f003533c6c0 1 -- start start 2026-02-13T02:22:27.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.007+0000 7fb1220c96c0 1 -- 10.20.193.27:0/3923550656 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb11c111000 msgr2=0x7fb11c111480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.007+0000 7fb1220c96c0 1 --2- 10.20.193.27:0/3923550656 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb11c111000 0x7fb11c111480 secure :-1 s=READY pgs=260 gs=2 cs=0 l=1 c_cookie=3896521382372230699 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb11000aa40 tx=0x7fb11002fe70 comp rx=0 tx=0).stop 2026-02-13T02:22:27.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.007+0000 7fb1220c96c0 1 -- 10.20.193.27:0/3923550656 shutdown_connections 2026-02-13T02:22:27.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.007+0000 7fb1220c96c0 1 --2- 10.20.193.27:0/3923550656 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb11c111000 0x7fb11c111480 unknown :-1 s=CLOSED pgs=260 gs=2 cs=0 l=1 c_cookie=3896521382372230699 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.007+0000 7fb1220c96c0 1 --2- 10.20.193.27:0/3923550656 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb11c10c540 0x7fb11c110a40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.007+0000 7fb1220c96c0 1 --2- 10.20.193.27:0/3923550656 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb11c10bc30 0x7fb11c10c000 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.008+0000 7fb1220c96c0 1 -- 10.20.193.27:0/3923550656 >> 10.20.193.27:0/3923550656 conn(0x7fb11c063bf0 msgr2=0x7fb11c064730 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.008+0000 7fb1220c96c0 1 -- 10.20.193.27:0/3923550656 shutdown_connections 2026-02-13T02:22:27.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.008+0000 7fb1220c96c0 1 -- 10.20.193.27:0/3923550656 wait complete. 2026-02-13T02:22:27.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.008+0000 7f003533c6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f003010f230 0x7f00300f4c40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.008+0000 7f003533c6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f0030115440 0x7f0030175ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.008+0000 7f003533c6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f00300f5270 0x7f00300fa540 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.008+0000 7f003533c6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f00300f58d0 con 0x7f003010f230 2026-02-13T02:22:27.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.008+0000 7f003533c6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f00300faa80 con 0x7f00300f5270 2026-02-13T02:22:27.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.008+0000 7f003533c6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f00300fac40 con 0x7f0030115440 2026-02-13T02:22:27.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.008+0000 7f002effd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f003010f230 0x7f00300f4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.008+0000 7f002effd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f003010f230 0x7f00300f4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55554/0 (socket says 10.20.193.27:55554) 2026-02-13T02:22:27.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.008+0000 7f002effd6c0 1 -- 10.20.193.27:0/1883138458 learned_addr learned my addr 10.20.193.27:0/1883138458 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:27.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.008+0000 7f002e7fc6c0 1 --2- 10.20.193.27:0/1883138458 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f0030115440 0x7f0030175ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.008+0000 7f002f7fe6c0 1 --2- 10.20.193.27:0/1883138458 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f00300f5270 0x7f00300fa540 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.014 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.008+0000 7fb1220c96c0 1 Processor -- start 2026-02-13T02:22:27.015 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.008+0000 7f002effd6c0 1 -- 10.20.193.27:0/1883138458 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f0030115440 msgr2=0x7f0030175ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.015 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.008+0000 7f002effd6c0 1 --2- 10.20.193.27:0/1883138458 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f0030115440 0x7f0030175ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.015 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.008+0000 7f002effd6c0 1 -- 10.20.193.27:0/1883138458 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f00300f5270 msgr2=0x7f00300fa540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.015 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.008+0000 7f002effd6c0 1 --2- 10.20.193.27:0/1883138458 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f00300f5270 0x7f00300fa540 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.015 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.008+0000 7f002effd6c0 1 -- 10.20.193.27:0/1883138458 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0018009590 con 0x7f003010f230 2026-02-13T02:22:27.015 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.008+0000 7f002effd6c0 1 --2- 10.20.193.27:0/1883138458 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f003010f230 0x7f00300f4c40 secure :-1 s=READY pgs=261 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0024009490 tx=0x7f002400bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.016 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.008+0000 7f000ffff6c0 1 -- 10.20.193.27:0/1883138458 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0024004a80 con 0x7f003010f230 2026-02-13T02:22:27.016 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.008+0000 7f003533c6c0 1 -- 10.20.193.27:0/1883138458 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00300faec0 con 0x7f003010f230 2026-02-13T02:22:27.016 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.008+0000 7f000ffff6c0 1 -- 10.20.193.27:0/1883138458 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f0024009030 con 0x7f003010f230 2026-02-13T02:22:27.016 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.008+0000 7f000ffff6c0 1 -- 10.20.193.27:0/1883138458 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f002400cdd0 con 0x7f003010f230 2026-02-13T02:22:27.016 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.008+0000 7f003533c6c0 1 -- 10.20.193.27:0/1883138458 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00300fb370 con 0x7f003010f230 2026-02-13T02:22:27.016 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.009+0000 7f003533c6c0 1 -- 10.20.193.27:0/1883138458 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0030052a50 con 0x7f003010f230 2026-02-13T02:22:27.018 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.009+0000 7fb1220c96c0 1 -- start start 2026-02-13T02:22:27.018 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.009+0000 7fb1220c96c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb11c10bc30 0x7fb11c122d00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.018 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.009+0000 7fb1220c96c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb11c10c540 0x7fb11c123240 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.018 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.009+0000 7fb1220c96c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb11c111000 0x7fb11c128600 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.018 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.009+0000 7fb1220c96c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c128c50 con 0x7fb11c10c540 2026-02-13T02:22:27.018 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.009+0000 7fb1220c96c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c128dc0 con 0x7fb11c10bc30 2026-02-13T02:22:27.019 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.009+0000 7fb1220c96c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c128f80 con 0x7fb11c111000 2026-02-13T02:22:27.019 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.009+0000 7fb11affd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb11c10c540 0x7fb11c123240 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.019 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.009+0000 7fb11affd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb11c10c540 0x7fb11c123240 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55566/0 (socket says 10.20.193.27:55566) 2026-02-13T02:22:27.019 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.009+0000 7fb11affd6c0 1 -- 10.20.193.27:0/3115177232 learned_addr learned my addr 10.20.193.27:0/3115177232 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:27.019 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.009+0000 7fb11bfff6c0 1 --2- 10.20.193.27:0/3115177232 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb11c111000 0x7fb11c128600 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.019 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.009+0000 7fb11affd6c0 1 -- 10.20.193.27:0/3115177232 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb11c111000 msgr2=0x7fb11c128600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.019 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.009+0000 7fb11affd6c0 1 --2- 10.20.193.27:0/3115177232 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb11c111000 0x7fb11c128600 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.019 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.009+0000 7fb11affd6c0 1 -- 10.20.193.27:0/3115177232 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb11c10bc30 msgr2=0x7fb11c122d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.019 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.009+0000 7fb11b7fe6c0 1 --2- 10.20.193.27:0/3115177232 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb11c10bc30 0x7fb11c122d00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.019 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.009+0000 7fb11affd6c0 1 --2- 10.20.193.27:0/3115177232 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb11c10bc30 0x7fb11c122d00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.009+0000 7fb11affd6c0 1 -- 10.20.193.27:0/3115177232 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11000a660 con 0x7fb11c10c540 2026-02-13T02:22:27.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.009+0000 7fb11b7fe6c0 1 --2- 10.20.193.27:0/3115177232 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb11c10bc30 0x7fb11c122d00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:22:27.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.010+0000 7fb11affd6c0 1 --2- 10.20.193.27:0/3115177232 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb11c10c540 0x7fb11c123240 secure :-1 s=READY pgs=262 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb10c0095e0 tx=0x7fb10c00bbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.010+0000 7fb118ff96c0 1 -- 10.20.193.27:0/3115177232 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb10c004a80 con 0x7fb11c10c540 2026-02-13T02:22:27.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.010+0000 7fb118ff96c0 1 -- 10.20.193.27:0/3115177232 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb10c009030 con 0x7fb11c10c540 2026-02-13T02:22:27.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.010+0000 7fb118ff96c0 1 -- 10.20.193.27:0/3115177232 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb10c00cd00 con 0x7fb11c10c540 2026-02-13T02:22:27.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.010+0000 7fb1220c96c0 1 -- 10.20.193.27:0/3115177232 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb11c129260 con 0x7fb11c10c540 2026-02-13T02:22:27.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.010+0000 7fb1220c96c0 1 -- 10.20.193.27:0/3115177232 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb11c169940 con 0x7fb11c10c540 2026-02-13T02:22:27.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.010+0000 7fb1220c96c0 1 -- 10.20.193.27:0/3115177232 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb11c052a50 con 0x7fb11c10c540 2026-02-13T02:22:27.021 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.011+0000 7f000ffff6c0 1 -- 10.20.193.27:0/1883138458 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f0024005000 con 0x7f003010f230 2026-02-13T02:22:27.021 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.012+0000 7f000ffff6c0 1 --2- 10.20.193.27:0/1883138458 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f00040b5710 0x7f00040d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.021 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.012+0000 7f000ffff6c0 1 -- 10.20.193.27:0/1883138458 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f00240e3b10 con 0x7f003010f230 2026-02-13T02:22:27.022 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.012+0000 7f002e7fc6c0 1 --2- 10.20.193.27:0/1883138458 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f00040b5710 0x7f00040d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.022 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.012+0000 7f000ffff6c0 1 --2- 10.20.193.27:0/1883138458 >> [v2:10.20.193.27:6802/832083434,v1:10.20.193.27:6803/832083434] conn(0x7f00040dccf0 0x7f00040fd100 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.022 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.012+0000 7f000ffff6c0 1 -- 10.20.193.27:0/1883138458 --> [v2:10.20.193.27:6802/832083434,v1:10.20.193.27:6803/832083434] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f00040fd7b0 con 0x7f00040dccf0 2026-02-13T02:22:27.022 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.012+0000 7f000ffff6c0 1 -- 10.20.193.27:0/1883138458 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f00240e3ec0 con 0x7f003010f230 2026-02-13T02:22:27.022 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.012+0000 7fb118ff96c0 1 -- 10.20.193.27:0/3115177232 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7fb10c004060 con 0x7fb11c10c540 2026-02-13T02:22:27.023 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.012+0000 7f002e7fc6c0 1 --2- 10.20.193.27:0/1883138458 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f00040b5710 0x7f00040d5bc0 secure :-1 s=READY pgs=185 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0018005b40 tx=0x7f00180023a0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.023 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.013+0000 7fb118ff96c0 1 --2- 10.20.193.27:0/3115177232 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb0f00b5710 0x7fb0f00d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.023 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.013+0000 7f002f7fe6c0 1 --2- 10.20.193.27:0/1883138458 >> [v2:10.20.193.27:6802/832083434,v1:10.20.193.27:6803/832083434] conn(0x7f00040dccf0 0x7f00040fd100 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.024 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.013+0000 7fb118ff96c0 1 -- 10.20.193.27:0/3115177232 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7fb10c0e39f0 con 0x7fb11c10c540 2026-02-13T02:22:27.024 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.013+0000 7fb11b7fe6c0 1 --2- 10.20.193.27:0/3115177232 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb0f00b5710 0x7fb0f00d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.024 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.013+0000 7fb118ff96c0 1 --2- 10.20.193.27:0/3115177232 >> [v2:10.20.193.28:6808/711804032,v1:10.20.193.28:6809/711804032] conn(0x7fb0f00dccf0 0x7fb0f00fd100 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.024 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.013+0000 7fb118ff96c0 1 -- 10.20.193.27:0/3115177232 --> [v2:10.20.193.28:6808/711804032,v1:10.20.193.28:6809/711804032] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb0f00fd7b0 con 0x7fb0f00dccf0 2026-02-13T02:22:27.024 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.013+0000 7fb118ff96c0 1 -- 10.20.193.27:0/3115177232 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fb10c0e3e10 con 0x7fb11c10c540 2026-02-13T02:22:27.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.013+0000 7f002f7fe6c0 1 --2- 10.20.193.27:0/1883138458 >> [v2:10.20.193.27:6802/832083434,v1:10.20.193.27:6803/832083434] conn(0x7f00040dccf0 0x7f00040fd100 crc :-1 s=READY pgs=7 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.013+0000 7fb11bfff6c0 1 --2- 10.20.193.27:0/3115177232 >> [v2:10.20.193.28:6808/711804032,v1:10.20.193.28:6809/711804032] conn(0x7fb0f00dccf0 0x7fb0f00fd100 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.013+0000 7fb11b7fe6c0 1 --2- 10.20.193.27:0/3115177232 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb0f00b5710 0x7fb0f00d5bc0 secure :-1 s=READY pgs=186 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb108004750 tx=0x7fb108005f10 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.026 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.013+0000 7f000ffff6c0 1 -- 10.20.193.27:0/1883138458 <== osd.0 v2:10.20.193.27:6802/832083434 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33498 (crc 0 0 0) 0x7f00040fd7b0 con 0x7f00040dccf0 2026-02-13T02:22:27.026 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.013+0000 7fb11bfff6c0 1 --2- 10.20.193.27:0/3115177232 >> [v2:10.20.193.28:6808/711804032,v1:10.20.193.28:6809/711804032] conn(0x7fb0f00dccf0 0x7fb0f00fd100 crc :-1 s=READY pgs=7 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.026 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.014+0000 7fb118ff96c0 1 -- 10.20.193.27:0/3115177232 <== osd.4 v2:10.20.193.28:6808/711804032 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33498 (crc 0 0 0) 0x7fb0f00fd7b0 con 0x7fb0f00dccf0 2026-02-13T02:22:27.027 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.015+0000 7fcb0170c6c0 1 -- 10.20.193.27:0/975071227 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcafc10cf80 msgr2=0x7fcafc10d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.015+0000 7fcb0170c6c0 1 --2- 10.20.193.27:0/975071227 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcafc10cf80 0x7fcafc10d350 secure :-1 s=READY pgs=264 gs=1 cs=0 l=1 c_cookie=2703948860509470793 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcaf00099a0 tx=0x7fcaf002ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:27.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.015+0000 7fcb0170c6c0 1 -- 10.20.193.27:0/975071227 shutdown_connections 2026-02-13T02:22:27.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.015+0000 7fcb0170c6c0 1 --2- 10.20.193.27:0/975071227 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fcafc113190 0x7fcafc173b30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.015+0000 7fcb0170c6c0 1 --2- 10.20.193.27:0/975071227 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcafc10d970 0x7fcafc1129b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.015+0000 7fcb0170c6c0 1 --2- 10.20.193.27:0/975071227 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcafc10cf80 0x7fcafc10d350 unknown :-1 s=CLOSED pgs=264 gs=1 cs=0 l=1 c_cookie=2703948860509470793 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.015+0000 7fcb0170c6c0 1 -- 10.20.193.27:0/975071227 >> 10.20.193.27:0/975071227 conn(0x7fcafc0ea880 msgr2=0x7fcafc10ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.015+0000 7fcb0170c6c0 1 -- 10.20.193.27:0/975071227 shutdown_connections 2026-02-13T02:22:27.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.015+0000 7fcb0170c6c0 1 -- 10.20.193.27:0/975071227 wait complete. 2026-02-13T02:22:27.029 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.015+0000 7f97b31016c0 1 -- 10.20.193.27:0/2201720813 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f97ac115440 msgr2=0x7f97ac175de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.029 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.015+0000 7f97b31016c0 1 --2- 10.20.193.27:0/2201720813 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f97ac115440 0x7f97ac175de0 secure :-1 s=READY pgs=263 gs=3 cs=0 l=1 c_cookie=9753932648395413673 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f97a800a9b0 tx=0x7f97a802fd20 comp rx=0 tx=0).stop 2026-02-13T02:22:27.029 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.015+0000 7f97b31016c0 1 -- 10.20.193.27:0/2201720813 shutdown_connections 2026-02-13T02:22:27.029 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.015+0000 7f97b31016c0 1 --2- 10.20.193.27:0/2201720813 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f97ac115440 0x7f97ac175de0 unknown :-1 s=CLOSED pgs=263 gs=3 cs=0 l=1 c_cookie=9753932648395413673 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.029 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.015+0000 7f97b31016c0 1 --2- 10.20.193.27:0/2201720813 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f97ac10fc20 0x7f97ac114c60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.029 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.015+0000 7f97b31016c0 1 --2- 10.20.193.27:0/2201720813 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f97ac10f230 0x7f97ac10f600 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.030 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.015+0000 7f97b31016c0 1 -- 10.20.193.27:0/2201720813 >> 10.20.193.27:0/2201720813 conn(0x7f97ac0ecb30 msgr2=0x7f97ac10cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.030 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.015+0000 7f97b31016c0 1 -- 10.20.193.27:0/2201720813 shutdown_connections 2026-02-13T02:22:27.030 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.015+0000 7f97b31016c0 1 -- 10.20.193.27:0/2201720813 wait complete. 2026-02-13T02:22:27.031 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.015+0000 7fcb0170c6c0 1 Processor -- start 2026-02-13T02:22:27.031 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.015+0000 7fcb0170c6c0 1 -- start start 2026-02-13T02:22:27.031 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.015+0000 7fcb0170c6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcafc10cf80 0x7fcafc11aff0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.031 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.016+0000 7fcb0170c6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fcafc113190 0x7fcafc11b530 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.031 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.016+0000 7fcb0170c6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcafc1208f0 0x7fcafc120d90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.031 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.016+0000 7fcb0170c6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc121300 con 0x7fcafc1208f0 2026-02-13T02:22:27.031 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.016+0000 7fcb0170c6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc121470 con 0x7fcafc10cf80 2026-02-13T02:22:27.031 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.016+0000 7fcb0170c6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc121630 con 0x7fcafc113190 2026-02-13T02:22:27.031 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.016+0000 7fcafb7fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcafc1208f0 0x7fcafc120d90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.032 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.016+0000 7fcafb7fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcafc1208f0 0x7fcafc120d90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55604/0 (socket says 10.20.193.27:55604) 2026-02-13T02:22:27.032 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.016+0000 7f167a2db6c0 1 -- 10.20.193.27:0/384798979 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f167410be20 msgr2=0x7f167410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.032 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.016+0000 7f167a2db6c0 1 --2- 10.20.193.27:0/384798979 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f167410be20 0x7f167410d950 secure :-1 s=READY pgs=265 gs=1 cs=0 l=1 c_cookie=5709432475186530619 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f16640075d0 tx=0x7f166402fde0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.033 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.016+0000 7fcafb7fe6c0 1 -- 10.20.193.27:0/2280556496 learned_addr learned my addr 10.20.193.27:0/2280556496 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:27.033 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.016+0000 7fcafa7fc6c0 1 --2- 10.20.193.27:0/2280556496 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fcafc113190 0x7fcafc11b530 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.034 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.016+0000 7fcafb7fe6c0 1 -- 10.20.193.27:0/2280556496 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fcafc113190 msgr2=0x7fcafc11b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.034 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.016+0000 7fcafb7fe6c0 1 --2- 10.20.193.27:0/2280556496 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fcafc113190 0x7fcafc11b530 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.034 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.016+0000 7fcafb7fe6c0 1 -- 10.20.193.27:0/2280556496 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcafc10cf80 msgr2=0x7fcafc11aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.034 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.016+0000 7fcafb7fe6c0 1 --2- 10.20.193.27:0/2280556496 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcafc10cf80 0x7fcafc11aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.034 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.016+0000 7fcafb7fe6c0 1 -- 10.20.193.27:0/2280556496 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf0009590 con 0x7fcafc1208f0 2026-02-13T02:22:27.034 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.016+0000 7fcafa7fc6c0 1 --2- 10.20.193.27:0/2280556496 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fcafc113190 0x7fcafc11b530 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:22:27.034 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.016+0000 7fcafb7fe6c0 1 --2- 10.20.193.27:0/2280556496 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcafc1208f0 0x7fcafc120d90 secure :-1 s=READY pgs=266 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcaec006ea0 tx=0x7fcaec00ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.034 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.016+0000 7fcadbfff6c0 1 -- 10.20.193.27:0/2280556496 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fcaec003c40 con 0x7fcafc1208f0 2026-02-13T02:22:27.034 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.016+0000 7fcb0170c6c0 1 -- 10.20.193.27:0/2280556496 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcafc060640 con 0x7fcafc1208f0 2026-02-13T02:22:27.034 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.016+0000 7fcadbfff6c0 1 -- 10.20.193.27:0/2280556496 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fcaec003da0 con 0x7fcafc1208f0 2026-02-13T02:22:27.035 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.016+0000 7f97b31016c0 1 Processor -- start 2026-02-13T02:22:27.035 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.016+0000 7f97b31016c0 1 -- start start 2026-02-13T02:22:27.036 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.016+0000 7fcadbfff6c0 1 -- 10.20.193.27:0/2280556496 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fcaec0081c0 con 0x7fcafc1208f0 2026-02-13T02:22:27.036 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.016+0000 7fcb0170c6c0 1 -- 10.20.193.27:0/2280556496 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcafc060ad0 con 0x7fcafc1208f0 2026-02-13T02:22:27.036 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.017+0000 7fcb0170c6c0 1 -- 10.20.193.27:0/2280556496 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcafc052a50 con 0x7fcafc1208f0 2026-02-13T02:22:27.038 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.017+0000 7f97b31016c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f97ac10f230 0x7f97ac11d280 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.038 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.017+0000 7f97b31016c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f97ac115440 0x7f97ac11d7c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.038 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.017+0000 7f97b0e726c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f97ac10f230 0x7f97ac11d280 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.038 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.017+0000 7f97b0e726c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f97ac10f230 0x7f97ac11d280 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55612/0 (socket says 10.20.193.27:55612) 2026-02-13T02:22:27.039 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.017+0000 7f97b0e726c0 1 -- 10.20.193.27:0/784936572 learned_addr learned my addr 10.20.193.27:0/784936572 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:27.039 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.017+0000 7f97b31016c0 1 --2- 10.20.193.27:0/784936572 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f97ac122b80 0x7f97ac123020 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.039 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.017+0000 7f97b31016c0 1 -- 10.20.193.27:0/784936572 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f97ac123590 con 0x7f97ac10f230 2026-02-13T02:22:27.039 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.017+0000 7f97b31016c0 1 -- 10.20.193.27:0/784936572 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f97ac123700 con 0x7f97ac122b80 2026-02-13T02:22:27.039 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.017+0000 7f97b31016c0 1 -- 10.20.193.27:0/784936572 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f97ac1238c0 con 0x7f97ac115440 2026-02-13T02:22:27.039 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.017+0000 7f97a3fff6c0 1 --2- 10.20.193.27:0/784936572 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f97ac115440 0x7f97ac11d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.039 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.017+0000 7f97b16736c0 1 --2- 10.20.193.27:0/784936572 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f97ac122b80 0x7f97ac123020 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.039 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.017+0000 7f97b0e726c0 1 -- 10.20.193.27:0/784936572 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f97ac115440 msgr2=0x7f97ac11d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.039 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.017+0000 7f97b0e726c0 1 --2- 10.20.193.27:0/784936572 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f97ac115440 0x7f97ac11d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.039 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.017+0000 7f97b0e726c0 1 -- 10.20.193.27:0/784936572 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f97ac122b80 msgr2=0x7f97ac123020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.039 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.018+0000 7f97b0e726c0 1 --2- 10.20.193.27:0/784936572 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f97ac122b80 0x7f97ac123020 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.040 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.018+0000 7f167a2db6c0 1 -- 10.20.193.27:0/384798979 shutdown_connections 2026-02-13T02:22:27.040 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.018+0000 7f167a2db6c0 1 --2- 10.20.193.27:0/384798979 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f167410e130 0x7f1674171d10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.040 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.018+0000 7f167a2db6c0 1 --2- 10.20.193.27:0/384798979 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f167410be20 0x7f167410d950 unknown :-1 s=CLOSED pgs=265 gs=1 cs=0 l=1 c_cookie=5709432475186530619 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.040 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.018+0000 7f167a2db6c0 1 --2- 10.20.193.27:0/384798979 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f167410b430 0x7f167410b800 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.040 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.018+0000 7f167a2db6c0 1 -- 10.20.193.27:0/384798979 >> 10.20.193.27:0/384798979 conn(0x7f1674063af0 msgr2=0x7f16740642b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.040 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.018+0000 7f167a2db6c0 1 -- 10.20.193.27:0/384798979 shutdown_connections 2026-02-13T02:22:27.041 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.018+0000 7f97b0e726c0 1 -- 10.20.193.27:0/784936572 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a800a5b0 con 0x7f97ac10f230 2026-02-13T02:22:27.042 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.018+0000 7f97a3fff6c0 1 --2- 10.20.193.27:0/784936572 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f97ac115440 0x7f97ac11d7c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-13T02:22:27.042 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.018+0000 7f97b16736c0 1 --2- 10.20.193.27:0/784936572 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f97ac122b80 0x7f97ac123020 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-13T02:22:27.042 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.018+0000 7f97b0e726c0 1 --2- 10.20.193.27:0/784936572 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f97ac10f230 0x7f97ac11d280 secure :-1 s=READY pgs=267 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f979c009490 tx=0x7f979c00bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.042 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.018+0000 7f97a1ffb6c0 1 -- 10.20.193.27:0/784936572 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f979c004a80 con 0x7f97ac10f230 2026-02-13T02:22:27.042 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.018+0000 7f97a1ffb6c0 1 -- 10.20.193.27:0/784936572 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f979c009030 con 0x7f97ac10f230 2026-02-13T02:22:27.042 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.018+0000 7f97a1ffb6c0 1 -- 10.20.193.27:0/784936572 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f979c00ce30 con 0x7f97ac10f230 2026-02-13T02:22:27.042 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.018+0000 7f167a2db6c0 1 -- 10.20.193.27:0/384798979 wait complete. 2026-02-13T02:22:27.042 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.018+0000 7f167a2db6c0 1 Processor -- start 2026-02-13T02:22:27.043 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.018+0000 7f97b31016c0 1 -- 10.20.193.27:0/784936572 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97ac163d70 con 0x7f97ac10f230 2026-02-13T02:22:27.043 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.018+0000 7f167a2db6c0 1 -- start start 2026-02-13T02:22:27.043 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.018+0000 7f97b31016c0 1 -- 10.20.193.27:0/784936572 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97ac164200 con 0x7f97ac10f230 2026-02-13T02:22:27.044 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.018+0000 7f167a2db6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f167410b430 0x7f1674059b20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.044 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.018+0000 7f167a2db6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f167410e130 0x7f1674171c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.044 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.018+0000 7fcadbfff6c0 1 -- 10.20.193.27:0/2280556496 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7fcaec00d590 con 0x7fcafc1208f0 2026-02-13T02:22:27.044 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.019+0000 7f97b31016c0 1 -- 10.20.193.27:0/784936572 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f97ac052a50 con 0x7f97ac10f230 2026-02-13T02:22:27.045 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.019+0000 7fcadbfff6c0 1 --2- 10.20.193.27:0/2280556496 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fcad00b5510 0x7fcad00d59c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.045 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.019+0000 7fcadbfff6c0 1 -- 10.20.193.27:0/2280556496 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7fcaec0e4d60 con 0x7fcafc1208f0 2026-02-13T02:22:27.045 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.019+0000 7fcadbfff6c0 1 --2- 10.20.193.27:0/2280556496 >> [v2:10.20.193.107:6800/4075625103,v1:10.20.193.107:6801/4075625103] conn(0x7fcad00dc980 0x7fcad00fcd90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.045 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.019+0000 7fcadbfff6c0 1 -- 10.20.193.27:0/2280556496 --> [v2:10.20.193.107:6800/4075625103,v1:10.20.193.107:6801/4075625103] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcad00fd440 con 0x7fcad00dc980 2026-02-13T02:22:27.046 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.018+0000 7f1673fff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f167410b430 0x7f1674059b20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.046 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.018+0000 7f1673fff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f167410b430 0x7f1674059b20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55624/0 (socket says 10.20.193.27:55624) 2026-02-13T02:22:27.046 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.018+0000 7f1673fff6c0 1 -- 10.20.193.27:0/2091612343 learned_addr learned my addr 10.20.193.27:0/2091612343 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:27.046 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.019+0000 7f167a2db6c0 1 --2- 10.20.193.27:0/2091612343 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f167405a060 0x7f167405a500 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.046 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.019+0000 7f167a2db6c0 1 -- 10.20.193.27:0/2091612343 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f16740fc5f0 con 0x7f167410b430 2026-02-13T02:22:27.046 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.019+0000 7f167a2db6c0 1 -- 10.20.193.27:0/2091612343 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f16740fc760 con 0x7f167405a060 2026-02-13T02:22:27.046 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.019+0000 7f167a2db6c0 1 -- 10.20.193.27:0/2091612343 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f16740fc920 con 0x7f167410e130 2026-02-13T02:22:27.046 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.019+0000 7f1673fff6c0 1 -- 10.20.193.27:0/2091612343 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f167410e130 msgr2=0x7f1674171c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.046 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.019+0000 7f1673fff6c0 1 --2- 10.20.193.27:0/2091612343 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f167410e130 0x7f1674171c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.046 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.019+0000 7f1673fff6c0 1 -- 10.20.193.27:0/2091612343 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f167405a060 msgr2=0x7f167405a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.046 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.019+0000 7f1673fff6c0 1 --2- 10.20.193.27:0/2091612343 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f167405a060 0x7f167405a500 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.046 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.019+0000 7f1673fff6c0 1 -- 10.20.193.27:0/2091612343 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16640071c0 con 0x7f167410b430 2026-02-13T02:22:27.046 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.019+0000 7f1673fff6c0 1 --2- 10.20.193.27:0/2091612343 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f167410b430 0x7f1674059b20 secure :-1 s=READY pgs=268 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f166000b800 tx=0x7f166000bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.047 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.019+0000 7f16717fa6c0 1 -- 10.20.193.27:0/2091612343 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1660004a80 con 0x7f167410b430 2026-02-13T02:22:27.047 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.019+0000 7f16717fa6c0 1 -- 10.20.193.27:0/2091612343 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1660009030 con 0x7f167410b430 2026-02-13T02:22:27.047 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.019+0000 7f167a2db6c0 1 -- 10.20.193.27:0/2091612343 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16740fcc00 con 0x7f167410b430 2026-02-13T02:22:27.047 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.019+0000 7f16717fa6c0 1 -- 10.20.193.27:0/2091612343 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f16600071c0 con 0x7f167410b430 2026-02-13T02:22:27.047 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.019+0000 7f167a2db6c0 1 -- 10.20.193.27:0/2091612343 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16740fd130 con 0x7f167410b430 2026-02-13T02:22:27.047 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.019+0000 7fcadbfff6c0 1 -- 10.20.193.27:0/2280556496 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fcaec0e5180 con 0x7fcafc1208f0 2026-02-13T02:22:27.047 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.019+0000 7fcafa7fc6c0 1 --2- 10.20.193.27:0/2280556496 >> [v2:10.20.193.107:6800/4075625103,v1:10.20.193.107:6801/4075625103] conn(0x7fcad00dc980 0x7fcad00fcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.048 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.020+0000 7f167a2db6c0 1 -- 10.20.193.27:0/2091612343 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1674052a50 con 0x7f167410b430 2026-02-13T02:22:27.048 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.020+0000 7fcafa7fc6c0 1 --2- 10.20.193.27:0/2280556496 >> [v2:10.20.193.107:6800/4075625103,v1:10.20.193.107:6801/4075625103] conn(0x7fcad00dc980 0x7fcad00fcd90 crc :-1 s=READY pgs=9 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.048 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.021+0000 7fcafaffd6c0 1 --2- 10.20.193.27:0/2280556496 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fcad00b5510 0x7fcad00d59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.048 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.021+0000 7f97a1ffb6c0 1 -- 10.20.193.27:0/784936572 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f979c004060 con 0x7f97ac10f230 2026-02-13T02:22:27.049 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.021+0000 7fcafaffd6c0 1 --2- 10.20.193.27:0/2280556496 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fcad00b5510 0x7fcad00d59c0 secure :-1 s=READY pgs=187 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcaf002f230 tx=0x7fcaf0005bc0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.049 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.021+0000 7f97a1ffb6c0 1 --2- 10.20.193.27:0/784936572 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f97880b5690 0x7f97880d5b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.049 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.021+0000 7f97a1ffb6c0 1 -- 10.20.193.27:0/784936572 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f979c0e3ad0 con 0x7f97ac10f230 2026-02-13T02:22:27.049 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.022+0000 7f97a1ffb6c0 1 --2- 10.20.193.27:0/784936572 >> [v2:10.20.193.28:6800/4070516162,v1:10.20.193.28:6801/4070516162] conn(0x7f97880dcc70 0x7f97880fd080 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.049 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.022+0000 7f97a1ffb6c0 1 -- 10.20.193.27:0/784936572 --> [v2:10.20.193.28:6800/4070516162,v1:10.20.193.28:6801/4070516162] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f97880fd730 con 0x7f97880dcc70 2026-02-13T02:22:27.050 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.022+0000 7f97a1ffb6c0 1 -- 10.20.193.27:0/784936572 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f979c0e3ef0 con 0x7f97ac10f230 2026-02-13T02:22:27.050 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.022+0000 7f97b16736c0 1 --2- 10.20.193.27:0/784936572 >> [v2:10.20.193.28:6800/4070516162,v1:10.20.193.28:6801/4070516162] conn(0x7f97880dcc70 0x7f97880fd080 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.050 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.022+0000 7f16717fa6c0 1 -- 10.20.193.27:0/2091612343 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f1660004050 con 0x7f167410b430 2026-02-13T02:22:27.050 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.022+0000 7fcadbfff6c0 1 -- 10.20.193.27:0/2280556496 <== osd.2 v2:10.20.193.107:6800/4075625103 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33498 (crc 0 0 0) 0x7fcad00fd440 con 0x7fcad00dc980 2026-02-13T02:22:27.050 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.022+0000 7f97b16736c0 1 --2- 10.20.193.27:0/784936572 >> [v2:10.20.193.28:6800/4070516162,v1:10.20.193.28:6801/4070516162] conn(0x7f97880dcc70 0x7f97880fd080 crc :-1 s=READY pgs=9 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.051 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.022+0000 7fb1220c96c0 1 -- 10.20.193.27:0/3115177232 --> [v2:10.20.193.28:6808/711804032,v1:10.20.193.28:6809/711804032] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb11c10fa00 con 0x7fb0f00dccf0 2026-02-13T02:22:27.051 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.022+0000 7fb118ff96c0 1 -- 10.20.193.27:0/3115177232 <== osd.4 v2:10.20.193.28:6808/711804032 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb11c10fa00 con 0x7fb0f00dccf0 2026-02-13T02:22:27.051 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.022+0000 7f16717fa6c0 1 --2- 10.20.193.27:0/2091612343 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f16480b5710 0x7f16480d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.051 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.023+0000 7f16717fa6c0 1 -- 10.20.193.27:0/2091612343 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f16600e3aa0 con 0x7f167410b430 2026-02-13T02:22:27.052 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.023+0000 7f16717fa6c0 1 --2- 10.20.193.27:0/2091612343 >> [v2:10.20.193.27:6810/382071678,v1:10.20.193.27:6811/382071678] conn(0x7f16480dccf0 0x7f16480fd100 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.052 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.023+0000 7f16717fa6c0 1 -- 10.20.193.27:0/2091612343 --> [v2:10.20.193.27:6810/382071678,v1:10.20.193.27:6811/382071678] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f16480fd7b0 con 0x7f16480dccf0 2026-02-13T02:22:27.052 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.023+0000 7f16717fa6c0 1 -- 10.20.193.27:0/2091612343 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f16600e3ec0 con 0x7f167410b430 2026-02-13T02:22:27.052 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.023+0000 7fb1220c96c0 1 -- 10.20.193.27:0/3115177232 >> [v2:10.20.193.28:6808/711804032,v1:10.20.193.28:6809/711804032] conn(0x7fb0f00dccf0 msgr2=0x7fb0f00fd100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.053 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.023+0000 7f97a1ffb6c0 1 -- 10.20.193.27:0/784936572 <== osd.1 v2:10.20.193.28:6800/4070516162 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33498 (crc 0 0 0) 0x7f97880fd730 con 0x7f97880dcc70 2026-02-13T02:22:27.053 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.023+0000 7f167884d6c0 1 --2- 10.20.193.27:0/2091612343 >> [v2:10.20.193.27:6810/382071678,v1:10.20.193.27:6811/382071678] conn(0x7f16480dccf0 0x7f16480fd100 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.053 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.023+0000 7fb1220c96c0 1 --2- 10.20.193.27:0/3115177232 >> [v2:10.20.193.28:6808/711804032,v1:10.20.193.28:6809/711804032] conn(0x7fb0f00dccf0 0x7fb0f00fd100 crc :-1 s=READY pgs=7 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.053 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.023+0000 7fb1220c96c0 1 -- 10.20.193.27:0/3115177232 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb0f00b5710 msgr2=0x7fb0f00d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.053 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.023+0000 7fb1220c96c0 1 --2- 10.20.193.27:0/3115177232 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb0f00b5710 0x7fb0f00d5bc0 secure :-1 s=READY pgs=186 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb108004750 tx=0x7fb108005f10 comp rx=0 tx=0).stop 2026-02-13T02:22:27.054 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.023+0000 7fb1220c96c0 1 -- 10.20.193.27:0/3115177232 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb11c10c540 msgr2=0x7fb11c123240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.054 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.023+0000 7fb1220c96c0 1 --2- 10.20.193.27:0/3115177232 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb11c10c540 0x7fb11c123240 secure :-1 s=READY pgs=262 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb10c0095e0 tx=0x7fb10c00bbf0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.054 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.023+0000 7fb11bfff6c0 1 -- 10.20.193.27:0/3115177232 reap_dead start 2026-02-13T02:22:27.054 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.023+0000 7fb1220c96c0 1 -- 10.20.193.27:0/3115177232 shutdown_connections 2026-02-13T02:22:27.054 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.023+0000 7fb1220c96c0 1 -- 10.20.193.27:0/3115177232 >> 10.20.193.27:0/3115177232 conn(0x7fb11c063bf0 msgr2=0x7fb11c10e860 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.054 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.023+0000 7fb1220c96c0 1 -- 10.20.193.27:0/3115177232 shutdown_connections 2026-02-13T02:22:27.054 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.023+0000 7f167884d6c0 1 --2- 10.20.193.27:0/2091612343 >> [v2:10.20.193.27:6810/382071678,v1:10.20.193.27:6811/382071678] conn(0x7f16480dccf0 0x7f16480fd100 crc :-1 s=READY pgs=7 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.054 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.023+0000 7fb1220c96c0 1 -- 10.20.193.27:0/3115177232 wait complete. 2026-02-13T02:22:27.055 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.024+0000 7f16717fa6c0 1 -- 10.20.193.27:0/2091612343 <== osd.3 v2:10.20.193.27:6810/382071678 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33498 (crc 0 0 0) 0x7f16480fd7b0 con 0x7f16480dccf0 2026-02-13T02:22:27.055 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.024+0000 7f003533c6c0 1 -- 10.20.193.27:0/1883138458 --> [v2:10.20.193.27:6802/832083434,v1:10.20.193.27:6803/832083434] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f00301792c0 con 0x7f00040dccf0 2026-02-13T02:22:27.055 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.024+0000 7f000ffff6c0 1 -- 10.20.193.27:0/1883138458 <== osd.0 v2:10.20.193.27:6802/832083434 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f00301792c0 con 0x7f00040dccf0 2026-02-13T02:22:27.056 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.025+0000 7f003533c6c0 1 -- 10.20.193.27:0/1883138458 >> [v2:10.20.193.27:6802/832083434,v1:10.20.193.27:6803/832083434] conn(0x7f00040dccf0 msgr2=0x7f00040fd100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.056 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.025+0000 7f003533c6c0 1 --2- 10.20.193.27:0/1883138458 >> [v2:10.20.193.27:6802/832083434,v1:10.20.193.27:6803/832083434] conn(0x7f00040dccf0 0x7f00040fd100 crc :-1 s=READY pgs=7 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.056 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.025+0000 7f003533c6c0 1 -- 10.20.193.27:0/1883138458 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f00040b5710 msgr2=0x7f00040d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.056 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.025+0000 7f003533c6c0 1 --2- 10.20.193.27:0/1883138458 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f00040b5710 0x7f00040d5bc0 secure :-1 s=READY pgs=185 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0018005b40 tx=0x7f00180023a0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.056 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.025+0000 7f003533c6c0 1 -- 10.20.193.27:0/1883138458 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f003010f230 msgr2=0x7f00300f4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.056 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.025+0000 7f003533c6c0 1 --2- 10.20.193.27:0/1883138458 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f003010f230 0x7f00300f4c40 secure :-1 s=READY pgs=261 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0024009490 tx=0x7f002400bc50 comp rx=0 tx=0).stop 2026-02-13T02:22:27.056 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.025+0000 7f002f7fe6c0 1 -- 10.20.193.27:0/1883138458 reap_dead start 2026-02-13T02:22:27.056 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.025+0000 7f003533c6c0 1 -- 10.20.193.27:0/1883138458 shutdown_connections 2026-02-13T02:22:27.056 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.025+0000 7f003533c6c0 1 -- 10.20.193.27:0/1883138458 >> 10.20.193.27:0/1883138458 conn(0x7f00300ecb30 msgr2=0x7f003010d890 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.056 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.025+0000 7f003533c6c0 1 -- 10.20.193.27:0/1883138458 shutdown_connections 2026-02-13T02:22:27.056 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.025+0000 7f003533c6c0 1 -- 10.20.193.27:0/1883138458 wait complete. 2026-02-13T02:22:27.057 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.028+0000 7ff57f3a06c0 1 -- 10.20.193.27:0/2964344674 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff57810fc20 msgr2=0x7ff578114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.057 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.028+0000 7ff57f3a06c0 1 --2- 10.20.193.27:0/2964344674 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff57810fc20 0x7ff578114c60 secure :-1 s=READY pgs=269 gs=1 cs=0 l=1 c_cookie=4790377720789660762 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff56c0099a0 tx=0x7ff56c02ed70 comp rx=0 tx=0).stop 2026-02-13T02:22:27.057 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.028+0000 7ff57f3a06c0 1 -- 10.20.193.27:0/2964344674 shutdown_connections 2026-02-13T02:22:27.057 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.028+0000 7ff57f3a06c0 1 --2- 10.20.193.27:0/2964344674 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff578115440 0x7ff578175de0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.057 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.028+0000 7ff57f3a06c0 1 --2- 10.20.193.27:0/2964344674 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff57810fc20 0x7ff578114c60 unknown :-1 s=CLOSED pgs=269 gs=1 cs=0 l=1 c_cookie=4790377720789660762 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.057 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.028+0000 7ff57f3a06c0 1 --2- 10.20.193.27:0/2964344674 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff57810f230 0x7ff57810f600 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.057 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.028+0000 7ff57f3a06c0 1 -- 10.20.193.27:0/2964344674 >> 10.20.193.27:0/2964344674 conn(0x7ff5780ecb30 msgr2=0x7ff57810cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.058 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.028+0000 7ff57f3a06c0 1 -- 10.20.193.27:0/2964344674 shutdown_connections 2026-02-13T02:22:27.058 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.028+0000 7ff57f3a06c0 1 -- 10.20.193.27:0/2964344674 wait complete. 2026-02-13T02:22:27.058 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.029+0000 7ff57f3a06c0 1 Processor -- start 2026-02-13T02:22:27.058 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.029+0000 7ff57f3a06c0 1 -- start start 2026-02-13T02:22:27.058 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.029+0000 7ff57f3a06c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff57810f230 0x7ff57811c910 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.058 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.029+0000 7ff57f3a06c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff578115440 0x7ff57811ce50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.058 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.029+0000 7ff57f3a06c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff578122210 0x7ff5781226b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.058 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.029+0000 7ff57f3a06c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff578122c20 con 0x7ff578115440 2026-02-13T02:22:27.058 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.029+0000 7ff57f3a06c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff578122d90 con 0x7ff57810f230 2026-02-13T02:22:27.058 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.029+0000 7ff57f3a06c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff578122f50 con 0x7ff578122210 2026-02-13T02:22:27.058 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.029+0000 7ff57c9146c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff578115440 0x7ff57811ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.058 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.029+0000 7ff57c9146c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff578115440 0x7ff57811ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55660/0 (socket says 10.20.193.27:55660) 2026-02-13T02:22:27.058 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.029+0000 7ff57c9146c0 1 -- 10.20.193.27:0/3678529562 learned_addr learned my addr 10.20.193.27:0/3678529562 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:27.058 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.029+0000 7ff57d9166c0 1 --2- 10.20.193.27:0/3678529562 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff578122210 0x7ff5781226b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.058 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.029+0000 7fcb0170c6c0 1 -- 10.20.193.27:0/2280556496 --> [v2:10.20.193.107:6800/4075625103,v1:10.20.193.107:6801/4075625103] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcafc177030 con 0x7fcad00dc980 2026-02-13T02:22:27.059 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7efdda5dd6c0 1 -- 10.20.193.27:0/966252978 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7efdd410fbf0 msgr2=0x7efdd4114c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.059 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7efdda5dd6c0 1 --2- 10.20.193.27:0/966252978 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7efdd410fbf0 0x7efdd4114c30 secure :-1 s=READY pgs=270 gs=1 cs=0 l=1 c_cookie=10857175812954316835 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7efdc40099a0 tx=0x7efdc402ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:27.059 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7ff57c9146c0 1 -- 10.20.193.27:0/3678529562 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff578122210 msgr2=0x7ff5781226b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.059 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7ff57c9146c0 1 --2- 10.20.193.27:0/3678529562 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff578122210 0x7ff5781226b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.059 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7ff57c9146c0 1 -- 10.20.193.27:0/3678529562 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff57810f230 msgr2=0x7ff57811c910 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2026-02-13T02:22:27.059 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7ff57c9146c0 1 --2- 10.20.193.27:0/3678529562 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff57810f230 0x7ff57811c910 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.059 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7ff57c9146c0 1 -- 10.20.193.27:0/3678529562 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff56c009590 con 0x7ff578115440 2026-02-13T02:22:27.059 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7fcadbfff6c0 1 -- 10.20.193.27:0/2280556496 <== osd.2 v2:10.20.193.107:6800/4075625103 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcafc177030 con 0x7fcad00dc980 2026-02-13T02:22:27.060 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7ff57c9146c0 1 --2- 10.20.193.27:0/3678529562 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff578115440 0x7ff57811ce50 secure :-1 s=READY pgs=271 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff56c0023c0 tx=0x7ff56c02fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.060 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7efdda5dd6c0 1 -- 10.20.193.27:0/966252978 shutdown_connections 2026-02-13T02:22:27.060 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7efdda5dd6c0 1 --2- 10.20.193.27:0/966252978 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7efdd4115410 0x7efdd4175db0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.060 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7efdda5dd6c0 1 --2- 10.20.193.27:0/966252978 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7efdd410fbf0 0x7efdd4114c30 unknown :-1 s=CLOSED pgs=270 gs=1 cs=0 l=1 c_cookie=10857175812954316835 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.060 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7efdda5dd6c0 1 --2- 10.20.193.27:0/966252978 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7efdd410f200 0x7efdd410f5d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.060 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7efdda5dd6c0 1 -- 10.20.193.27:0/966252978 >> 10.20.193.27:0/966252978 conn(0x7efdd40ecb00 msgr2=0x7efdd410cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.060 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7ff5667fc6c0 1 -- 10.20.193.27:0/3678529562 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff56c002ae0 con 0x7ff578115440 2026-02-13T02:22:27.060 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7ff57f3a06c0 1 -- 10.20.193.27:0/3678529562 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5781633a0 con 0x7ff578115440 2026-02-13T02:22:27.060 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7ff57f3a06c0 1 -- 10.20.193.27:0/3678529562 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff578163820 con 0x7ff578115440 2026-02-13T02:22:27.061 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7ff5667fc6c0 1 -- 10.20.193.27:0/3678529562 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff56c007680 con 0x7ff578115440 2026-02-13T02:22:27.061 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7efdda5dd6c0 1 -- 10.20.193.27:0/966252978 shutdown_connections 2026-02-13T02:22:27.061 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7fcb0170c6c0 1 -- 10.20.193.27:0/2280556496 >> [v2:10.20.193.107:6800/4075625103,v1:10.20.193.107:6801/4075625103] conn(0x7fcad00dc980 msgr2=0x7fcad00fcd90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.061 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7fcb0170c6c0 1 --2- 10.20.193.27:0/2280556496 >> [v2:10.20.193.107:6800/4075625103,v1:10.20.193.107:6801/4075625103] conn(0x7fcad00dc980 0x7fcad00fcd90 crc :-1 s=READY pgs=9 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.061 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7efdda5dd6c0 1 -- 10.20.193.27:0/966252978 wait complete. 2026-02-13T02:22:27.062 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7fcb0170c6c0 1 -- 10.20.193.27:0/2280556496 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fcad00b5510 msgr2=0x7fcad00d59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.062 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7fcb0170c6c0 1 --2- 10.20.193.27:0/2280556496 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fcad00b5510 0x7fcad00d59c0 secure :-1 s=READY pgs=187 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcaf002f230 tx=0x7fcaf0005bc0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.062 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7fcb0170c6c0 1 -- 10.20.193.27:0/2280556496 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcafc1208f0 msgr2=0x7fcafc120d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.062 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7fcb0170c6c0 1 --2- 10.20.193.27:0/2280556496 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcafc1208f0 0x7fcafc120d90 secure :-1 s=READY pgs=266 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcaec006ea0 tx=0x7fcaec00ce50 comp rx=0 tx=0).stop 2026-02-13T02:22:27.062 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7fcafb7fe6c0 1 -- 10.20.193.27:0/2280556496 reap_dead start 2026-02-13T02:22:27.062 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7fcb0170c6c0 1 -- 10.20.193.27:0/2280556496 shutdown_connections 2026-02-13T02:22:27.062 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7fcb0170c6c0 1 -- 10.20.193.27:0/2280556496 >> 10.20.193.27:0/2280556496 conn(0x7fcafc0ea880 msgr2=0x7fcafc10ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.062 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7fcb0170c6c0 1 -- 10.20.193.27:0/2280556496 shutdown_connections 2026-02-13T02:22:27.062 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7fcb0170c6c0 1 -- 10.20.193.27:0/2280556496 wait complete. 2026-02-13T02:22:27.062 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7ff5667fc6c0 1 -- 10.20.193.27:0/3678529562 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff56c036590 con 0x7ff578115440 2026-02-13T02:22:27.062 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.030+0000 7ff57f3a06c0 1 -- 10.20.193.27:0/3678529562 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff578052a50 con 0x7ff578115440 2026-02-13T02:22:27.063 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.031+0000 7f16737fe6c0 1 --2- 10.20.193.27:0/2091612343 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f16480b5710 0x7f16480d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.063 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.031+0000 7efdda5dd6c0 1 Processor -- start 2026-02-13T02:22:27.063 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.031+0000 7efdda5dd6c0 1 -- start start 2026-02-13T02:22:27.063 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.031+0000 7f16737fe6c0 1 --2- 10.20.193.27:0/2091612343 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f16480b5710 0x7f16480d5bc0 secure :-1 s=READY pgs=189 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f16640302f0 tx=0x7f1664005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.063 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.031+0000 7f97a3fff6c0 1 --2- 10.20.193.27:0/784936572 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f97880b5690 0x7f97880d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.063 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.031+0000 7f97a3fff6c0 1 --2- 10.20.193.27:0/784936572 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f97880b5690 0x7f97880d5b40 secure :-1 s=READY pgs=188 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f97ac123c10 tx=0x7f9794009310 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.064 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.031+0000 7efdda5dd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7efdd410f200 0x7efdd40f4af0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.064 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.031+0000 7efdda5dd6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7efdd4115410 0x7efdd4175cb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.064 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.031+0000 7efdda5dd6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7efdd40f5120 0x7efdd40fa3a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.064 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.031+0000 7efdd3fff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7efdd410f200 0x7efdd40f4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.064 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.031+0000 7efdda5dd6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7efdd40f5810 con 0x7efdd410f200 2026-02-13T02:22:27.064 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.031+0000 7efdd3fff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7efdd410f200 0x7efdd40f4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55680/0 (socket says 10.20.193.27:55680) 2026-02-13T02:22:27.064 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.031+0000 7efdd3fff6c0 1 -- 10.20.193.27:0/922388294 learned_addr learned my addr 10.20.193.27:0/922388294 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:27.064 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.031+0000 7efdd8b4f6c0 1 --2- 10.20.193.27:0/922388294 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7efdd40f5120 0x7efdd40fa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.064 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.031+0000 7efdd37fe6c0 1 --2- 10.20.193.27:0/922388294 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7efdd4115410 0x7efdd4175cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.064 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.031+0000 7efdda5dd6c0 1 -- 10.20.193.27:0/922388294 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7efdd40fa8e0 con 0x7efdd40f5120 2026-02-13T02:22:27.064 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.031+0000 7efdda5dd6c0 1 -- 10.20.193.27:0/922388294 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7efdd40faaa0 con 0x7efdd4115410 2026-02-13T02:22:27.064 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7fb75aa186c0 1 -- 10.20.193.27:0/520939249 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb75410f230 msgr2=0x7fb75410f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.064 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7fb75aa186c0 1 --2- 10.20.193.27:0/520939249 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb75410f230 0x7fb75410f600 secure :-1 s=READY pgs=53 gs=1 cs=0 l=1 c_cookie=12201672893593796161 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb7400099a0 tx=0x7fb74002ed50 comp rx=0 tx=0).stop 2026-02-13T02:22:27.065 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7efdd3fff6c0 1 -- 10.20.193.27:0/922388294 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7efdd4115410 msgr2=0x7efdd4175cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.065 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7efdd3fff6c0 1 --2- 10.20.193.27:0/922388294 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7efdd4115410 0x7efdd4175cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.065 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7efdd3fff6c0 1 -- 10.20.193.27:0/922388294 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7efdd40f5120 msgr2=0x7efdd40fa3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.065 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7efdd3fff6c0 1 --2- 10.20.193.27:0/922388294 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7efdd40f5120 0x7efdd40fa3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.065 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7efdd3fff6c0 1 -- 10.20.193.27:0/922388294 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc4009590 con 0x7efdd410f200 2026-02-13T02:22:27.065 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7efdd8b4f6c0 1 --2- 10.20.193.27:0/922388294 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7efdd40f5120 0x7efdd40fa3a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-13T02:22:27.065 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7f167a2db6c0 1 -- 10.20.193.27:0/2091612343 --> [v2:10.20.193.27:6810/382071678,v1:10.20.193.27:6811/382071678] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1674173a50 con 0x7f16480dccf0 2026-02-13T02:22:27.065 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7f16717fa6c0 1 -- 10.20.193.27:0/2091612343 <== osd.3 v2:10.20.193.27:6810/382071678 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1674173a50 con 0x7f16480dccf0 2026-02-13T02:22:27.065 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7efdd3fff6c0 1 --2- 10.20.193.27:0/922388294 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7efdd410f200 0x7efdd40f4af0 secure :-1 s=READY pgs=272 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7efdc0005dc0 tx=0x7efdc000bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.066 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7fb75aa186c0 1 -- 10.20.193.27:0/520939249 shutdown_connections 2026-02-13T02:22:27.066 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7fb75aa186c0 1 --2- 10.20.193.27:0/520939249 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb754115440 0x7fb754175de0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.066 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7fb75aa186c0 1 --2- 10.20.193.27:0/520939249 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb75410fc20 0x7fb754114c60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.066 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7fb75aa186c0 1 --2- 10.20.193.27:0/520939249 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb75410f230 0x7fb75410f600 unknown :-1 s=CLOSED pgs=53 gs=1 cs=0 l=1 c_cookie=12201672893593796161 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.066 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7fb75aa186c0 1 -- 10.20.193.27:0/520939249 >> 10.20.193.27:0/520939249 conn(0x7fb7540ecb30 msgr2=0x7fb75410cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.066 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7f167a2db6c0 1 -- 10.20.193.27:0/2091612343 >> [v2:10.20.193.27:6810/382071678,v1:10.20.193.27:6811/382071678] conn(0x7f16480dccf0 msgr2=0x7f16480fd100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.066 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7f167a2db6c0 1 --2- 10.20.193.27:0/2091612343 >> [v2:10.20.193.27:6810/382071678,v1:10.20.193.27:6811/382071678] conn(0x7f16480dccf0 0x7f16480fd100 crc :-1 s=READY pgs=7 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.066 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7f167a2db6c0 1 -- 10.20.193.27:0/2091612343 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f16480b5710 msgr2=0x7f16480d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.066 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7f167a2db6c0 1 --2- 10.20.193.27:0/2091612343 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f16480b5710 0x7f16480d5bc0 secure :-1 s=READY pgs=189 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f16640302f0 tx=0x7f1664005910 comp rx=0 tx=0).stop 2026-02-13T02:22:27.066 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7f167a2db6c0 1 -- 10.20.193.27:0/2091612343 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f167410b430 msgr2=0x7f1674059b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.066 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7f167a2db6c0 1 --2- 10.20.193.27:0/2091612343 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f167410b430 0x7f1674059b20 secure :-1 s=READY pgs=268 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f166000b800 tx=0x7f166000bcd0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.067 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7f167884d6c0 1 -- 10.20.193.27:0/2091612343 reap_dead start 2026-02-13T02:22:27.067 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7fb75aa186c0 1 -- 10.20.193.27:0/520939249 shutdown_connections 2026-02-13T02:22:27.067 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7fb75aa186c0 1 -- 10.20.193.27:0/520939249 wait complete. 2026-02-13T02:22:27.067 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7f167a2db6c0 1 -- 10.20.193.27:0/2091612343 shutdown_connections 2026-02-13T02:22:27.067 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7f167a2db6c0 1 -- 10.20.193.27:0/2091612343 >> 10.20.193.27:0/2091612343 conn(0x7f1674063af0 msgr2=0x7f167410c940 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.067 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.033+0000 7f167a2db6c0 1 -- 10.20.193.27:0/2091612343 shutdown_connections 2026-02-13T02:22:27.067 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.033+0000 7f167a2db6c0 1 -- 10.20.193.27:0/2091612343 wait complete. 2026-02-13T02:22:27.067 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.033+0000 7fb75aa186c0 1 Processor -- start 2026-02-13T02:22:27.068 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7efdd17fa6c0 1 -- 10.20.193.27:0/922388294 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7efdc0004250 con 0x7efdd410f200 2026-02-13T02:22:27.068 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7efdd17fa6c0 1 -- 10.20.193.27:0/922388294 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7efdc0009070 con 0x7efdd410f200 2026-02-13T02:22:27.068 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7efdda5dd6c0 1 -- 10.20.193.27:0/922388294 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd40fad80 con 0x7efdd410f200 2026-02-13T02:22:27.068 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7efdda5dd6c0 1 -- 10.20.193.27:0/922388294 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd40fb200 con 0x7efdd410f200 2026-02-13T02:22:27.068 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.032+0000 7efdd17fa6c0 1 -- 10.20.193.27:0/922388294 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7efdc000cde0 con 0x7efdd410f200 2026-02-13T02:22:27.068 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.033+0000 7efdda5dd6c0 1 -- 10.20.193.27:0/922388294 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efdd4052a50 con 0x7efdd410f200 2026-02-13T02:22:27.068 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.033+0000 7fb75aa186c0 1 -- start start 2026-02-13T02:22:27.068 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.033+0000 7fb75aa186c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb75410f230 0x7fb75411d0a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.068 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.033+0000 7f97b31016c0 1 -- 10.20.193.27:0/784936572 --> [v2:10.20.193.28:6800/4070516162,v1:10.20.193.28:6801/4070516162] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f97ac179300 con 0x7f97880dcc70 2026-02-13T02:22:27.069 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.033+0000 7fb75aa186c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb754115440 0x7fb75411d5e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.069 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.033+0000 7f97a1ffb6c0 1 -- 10.20.193.27:0/784936572 <== osd.1 v2:10.20.193.28:6800/4070516162 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f97ac179300 con 0x7f97880dcc70 2026-02-13T02:22:27.069 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.033+0000 7fb75aa186c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb7541229a0 0x7fb754122e40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.069 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.033+0000 7f2c9ae746c0 1 -- 10.20.193.27:0/139029200 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2c94112fc0 msgr2=0x7f2c9410b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.069 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.033+0000 7f2c9ae746c0 1 --2- 10.20.193.27:0/139029200 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2c94112fc0 0x7f2c9410b730 secure :-1 s=READY pgs=51 gs=2 cs=0 l=1 c_cookie=15471358429508562897 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2c7c009970 tx=0x7f2c7c02ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:27.069 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.033+0000 7fb75aa186c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb7541233b0 con 0x7fb75410f230 2026-02-13T02:22:27.070 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7fb75aa186c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb754123520 con 0x7fb754115440 2026-02-13T02:22:27.070 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7fb75aa186c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb7541236e0 con 0x7fb7541229a0 2026-02-13T02:22:27.070 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7f2c9ae746c0 1 -- 10.20.193.27:0/139029200 shutdown_connections 2026-02-13T02:22:27.070 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7f2c9ae746c0 1 --2- 10.20.193.27:0/139029200 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2c9410c4e0 0x7f2c9410c980 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.070 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7f2c9ae746c0 1 --2- 10.20.193.27:0/139029200 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2c94112fc0 0x7f2c9410b730 unknown :-1 s=CLOSED pgs=51 gs=2 cs=0 l=1 c_cookie=15471358429508562897 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.070 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7f2c9ae746c0 1 --2- 10.20.193.27:0/139029200 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2c94112b10 0x7f2c9410b1f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.070 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7f2c9ae746c0 1 -- 10.20.193.27:0/139029200 >> 10.20.193.27:0/139029200 conn(0x7f2c94063af0 msgr2=0x7f2c940642b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.070 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7f97b31016c0 1 -- 10.20.193.27:0/784936572 >> [v2:10.20.193.28:6800/4070516162,v1:10.20.193.28:6801/4070516162] conn(0x7f97880dcc70 msgr2=0x7f97880fd080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.070 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7f97b31016c0 1 --2- 10.20.193.27:0/784936572 >> [v2:10.20.193.28:6800/4070516162,v1:10.20.193.28:6801/4070516162] conn(0x7f97880dcc70 0x7f97880fd080 crc :-1 s=READY pgs=9 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.070 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7f2c9ae746c0 1 -- 10.20.193.27:0/139029200 shutdown_connections 2026-02-13T02:22:27.070 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7fb7537fe6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb754115440 0x7fb75411d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.070 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7fb7537fe6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb754115440 0x7fb75411d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.27:47852/0 (socket says 10.20.193.27:47852) 2026-02-13T02:22:27.071 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7f2c9ae746c0 1 -- 10.20.193.27:0/139029200 wait complete. 2026-02-13T02:22:27.071 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7fb7537fe6c0 1 -- 10.20.193.27:0/2757875259 learned_addr learned my addr 10.20.193.27:0/2757875259 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:27.071 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7fb753fff6c0 1 --2- 10.20.193.27:0/2757875259 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb75410f230 0x7fb75411d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.071 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7fb753fff6c0 1 -- 10.20.193.27:0/2757875259 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb7541229a0 msgr2=0x7fb754122e40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2026-02-13T02:22:27.071 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7fb753fff6c0 1 --2- 10.20.193.27:0/2757875259 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb7541229a0 0x7fb754122e40 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.072 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7fb753fff6c0 1 -- 10.20.193.27:0/2757875259 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb754115440 msgr2=0x7fb75411d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.072 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7fb753fff6c0 1 --2- 10.20.193.27:0/2757875259 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb754115440 0x7fb75411d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.072 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7fb753fff6c0 1 -- 10.20.193.27:0/2757875259 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb74800a5b0 con 0x7fb75410f230 2026-02-13T02:22:27.072 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7fb753fff6c0 1 --2- 10.20.193.27:0/2757875259 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb75410f230 0x7fb75411d0a0 secure :-1 s=READY pgs=273 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb7400028e0 tx=0x7fb740002910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.072 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7fb7517fa6c0 1 -- 10.20.193.27:0/2757875259 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb74002fcb0 con 0x7fb75410f230 2026-02-13T02:22:27.072 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7fb75aa186c0 1 -- 10.20.193.27:0/2757875259 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb740009590 con 0x7fb75410f230 2026-02-13T02:22:27.072 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7fb7517fa6c0 1 -- 10.20.193.27:0/2757875259 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb740007680 con 0x7fb75410f230 2026-02-13T02:22:27.072 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7fb7517fa6c0 1 -- 10.20.193.27:0/2757875259 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb7400362f0 con 0x7fb75410f230 2026-02-13T02:22:27.072 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7f2c9ae746c0 1 Processor -- start 2026-02-13T02:22:27.072 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7ff5667fc6c0 1 -- 10.20.193.27:0/3678529562 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7ff56c002c40 con 0x7ff578115440 2026-02-13T02:22:27.072 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7fcd434eb6c0 1 -- 10.20.193.27:0/3627362739 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcd3c10c4e0 msgr2=0x7fcd3c10c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.072 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7fcd434eb6c0 1 --2- 10.20.193.27:0/3627362739 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcd3c10c4e0 0x7fcd3c10c980 secure :-1 s=READY pgs=50 gs=2 cs=0 l=1 c_cookie=6903306281391371264 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcd3800aa40 tx=0x7fcd3802fde0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.072 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7fcd434eb6c0 1 -- 10.20.193.27:0/3627362739 shutdown_connections 2026-02-13T02:22:27.072 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7fcd434eb6c0 1 --2- 10.20.193.27:0/3627362739 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcd3c10c4e0 0x7fcd3c10c980 unknown :-1 s=CLOSED pgs=50 gs=2 cs=0 l=1 c_cookie=6903306281391371264 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.072 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7fcd434eb6c0 1 --2- 10.20.193.27:0/3627362739 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fcd3c112fc0 0x7fcd3c10b730 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.073 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7fcd434eb6c0 1 --2- 10.20.193.27:0/3627362739 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcd3c112b10 0x7fcd3c10b1f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.073 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7fcd434eb6c0 1 -- 10.20.193.27:0/3627362739 >> 10.20.193.27:0/3627362739 conn(0x7fcd3c063af0 msgr2=0x7fcd3c0642b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.073 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7fcd434eb6c0 1 -- 10.20.193.27:0/3627362739 shutdown_connections 2026-02-13T02:22:27.073 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7fcd434eb6c0 1 -- 10.20.193.27:0/3627362739 wait complete. 2026-02-13T02:22:27.073 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7fb75aa186c0 1 -- 10.20.193.27:0/2757875259 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb754163df0 con 0x7fb75410f230 2026-02-13T02:22:27.073 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7ff5667fc6c0 1 --2- 10.20.193.27:0/3678529562 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff5540b5790 0x7ff5540d5c40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.073 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7fcd434eb6c0 1 Processor -- start 2026-02-13T02:22:27.073 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7f2c9ae746c0 1 -- start start 2026-02-13T02:22:27.073 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7f2c9ae746c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2c9410c4e0 0x7f2c94104040 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.073 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7f2c9ae746c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2c94112b10 0x7f2c94104580 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.073 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7f2c9ae746c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2c94112fc0 0x7f2c94109940 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.074 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7fcd434eb6c0 1 -- start start 2026-02-13T02:22:27.074 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7f2c9ae746c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2c94109f90 con 0x7f2c9410c4e0 2026-02-13T02:22:27.074 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7f2c9ae746c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9410a100 con 0x7f2c94112fc0 2026-02-13T02:22:27.074 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7f2c9ae746c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9410a2c0 con 0x7f2c94112b10 2026-02-13T02:22:27.074 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7ff5667fc6c0 1 -- 10.20.193.27:0/3678529562 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7ff56c107cf0 con 0x7ff578115440 2026-02-13T02:22:27.074 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.034+0000 7efdd17fa6c0 1 -- 10.20.193.27:0/922388294 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7efdc00028c0 con 0x7efdd410f200 2026-02-13T02:22:27.074 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7efdd17fa6c0 1 --2- 10.20.193.27:0/922388294 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7efda80b5790 0x7efda80d5c40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.074 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7f2c8bfff6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2c94112b10 0x7f2c94104580 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.074 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7f2c8bfff6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2c94112b10 0x7f2c94104580 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.107:3300/0 says I am v2:10.20.193.27:43122/0 (socket says 10.20.193.27:43122) 2026-02-13T02:22:27.074 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7f2c8bfff6c0 1 -- 10.20.193.27:0/300191952 learned_addr learned my addr 10.20.193.27:0/300191952 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:27.075 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7f97b31016c0 1 -- 10.20.193.27:0/784936572 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f97880b5690 msgr2=0x7f97880d5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.075 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7f97b31016c0 1 --2- 10.20.193.27:0/784936572 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f97880b5690 0x7f97880d5b40 secure :-1 s=READY pgs=188 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f97ac123c10 tx=0x7f9794009310 comp rx=0 tx=0).stop 2026-02-13T02:22:27.075 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7f97b31016c0 1 -- 10.20.193.27:0/784936572 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f97ac10f230 msgr2=0x7f97ac11d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.075 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7f97b31016c0 1 --2- 10.20.193.27:0/784936572 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f97ac10f230 0x7f97ac11d280 secure :-1 s=READY pgs=267 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f979c009490 tx=0x7f979c00bca0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.075 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7f97b16736c0 1 -- 10.20.193.27:0/784936572 reap_dead start 2026-02-13T02:22:27.075 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7ff5667fc6c0 1 --2- 10.20.193.27:0/3678529562 >> [v2:10.20.193.28:6824/2255755797,v1:10.20.193.28:6825/2255755797] conn(0x7ff5540dcd70 0x7ff5540fd180 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.075 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7fcd434eb6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcd3c10c4e0 0x7fcd3c0f2960 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.075 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7ff5667fc6c0 1 -- 10.20.193.27:0/3678529562 --> [v2:10.20.193.28:6824/2255755797,v1:10.20.193.28:6825/2255755797] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff5540fd830 con 0x7ff5540dcd70 2026-02-13T02:22:27.075 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7ff5667fc6c0 1 -- 10.20.193.27:0/3678529562 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7ff56c108020 con 0x7ff578115440 2026-02-13T02:22:27.075 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7fcd434eb6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcd3c112b10 0x7fcd3c0f2ea0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.075 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7f2c8bfff6c0 1 -- 10.20.193.27:0/300191952 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2c94112fc0 msgr2=0x7f2c94109940 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2026-02-13T02:22:27.076 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7f2c8bfff6c0 1 --2- 10.20.193.27:0/300191952 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2c94112fc0 0x7f2c94109940 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.076 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7f2c8bfff6c0 1 -- 10.20.193.27:0/300191952 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2c9410c4e0 msgr2=0x7f2c94104040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-13T02:22:27.076 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7f2c8bfff6c0 1 --2- 10.20.193.27:0/300191952 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2c9410c4e0 0x7f2c94104040 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.076 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7f2c8bfff6c0 1 -- 10.20.193.27:0/300191952 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7c009590 con 0x7f2c94112b10 2026-02-13T02:22:27.076 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7f97b31016c0 1 -- 10.20.193.27:0/784936572 shutdown_connections 2026-02-13T02:22:27.076 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7f97b31016c0 1 -- 10.20.193.27:0/784936572 >> 10.20.193.27:0/784936572 conn(0x7f97ac0ecb30 msgr2=0x7f97ac10cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.076 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7ff57d9166c0 1 --2- 10.20.193.27:0/3678529562 >> [v2:10.20.193.28:6824/2255755797,v1:10.20.193.28:6825/2255755797] conn(0x7ff5540dcd70 0x7ff5540fd180 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.076 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7fcd40a5b6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcd3c112b10 0x7fcd3c0f2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.076 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7fcd40a5b6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcd3c112b10 0x7fcd3c0f2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55716/0 (socket says 10.20.193.27:55716) 2026-02-13T02:22:27.076 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7fcd40a5b6c0 1 -- 10.20.193.27:0/210307225 learned_addr learned my addr 10.20.193.27:0/210307225 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:27.076 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7fcd4125c6c0 1 --2- 10.20.193.27:0/210307225 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcd3c10c4e0 0x7fcd3c0f2960 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.076 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7f97b31016c0 1 -- 10.20.193.27:0/784936572 shutdown_connections 2026-02-13T02:22:27.077 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7f97b31016c0 1 -- 10.20.193.27:0/784936572 wait complete. 2026-02-13T02:22:27.077 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7fcd434eb6c0 1 --2- 10.20.193.27:0/210307225 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fcd3c112fc0 0x7fcd3c0f8260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.077 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7fcd434eb6c0 1 -- 10.20.193.27:0/210307225 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c0f88b0 con 0x7fcd3c112b10 2026-02-13T02:22:27.077 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7fcd434eb6c0 1 -- 10.20.193.27:0/210307225 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c0f8a20 con 0x7fcd3c10c4e0 2026-02-13T02:22:27.077 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7f2c8bfff6c0 1 --2- 10.20.193.27:0/300191952 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2c94112b10 0x7f2c94104580 secure :-1 s=READY pgs=54 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2c7c0098c0 tx=0x7f2c7c02fcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.077 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7fcd434eb6c0 1 -- 10.20.193.27:0/210307225 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c0f8be0 con 0x7fcd3c112fc0 2026-02-13T02:22:27.077 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7ff57d1156c0 1 --2- 10.20.193.27:0/3678529562 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff5540b5790 0x7ff5540d5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.077 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7efdd37fe6c0 1 --2- 10.20.193.27:0/922388294 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7efda80b5790 0x7efda80d5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.077 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7fb75aa186c0 1 -- 10.20.193.27:0/2757875259 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb754052a50 con 0x7fb75410f230 2026-02-13T02:22:27.078 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7f2c89ffb6c0 1 -- 10.20.193.27:0/300191952 <== mon.1 v2:10.20.193.107:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2c7c002ae0 con 0x7f2c94112b10 2026-02-13T02:22:27.078 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.036+0000 7ff57d9166c0 1 --2- 10.20.193.27:0/3678529562 >> [v2:10.20.193.28:6824/2255755797,v1:10.20.193.28:6825/2255755797] conn(0x7ff5540dcd70 0x7ff5540fd180 crc :-1 s=READY pgs=9 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.078 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.035+0000 7f2c89ffb6c0 1 -- 10.20.193.27:0/300191952 <== mon.1 v2:10.20.193.107:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f2c7c002c40 con 0x7f2c94112b10 2026-02-13T02:22:27.078 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.036+0000 7efdd17fa6c0 1 -- 10.20.193.27:0/922388294 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7efdc00e3a40 con 0x7efdd410f200 2026-02-13T02:22:27.078 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.036+0000 7fcd41a5d6c0 1 --2- 10.20.193.27:0/210307225 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fcd3c112fc0 0x7fcd3c0f8260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.078 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.036+0000 7fcd40a5b6c0 1 -- 10.20.193.27:0/210307225 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fcd3c112fc0 msgr2=0x7fcd3c0f8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.078 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.036+0000 7fcd40a5b6c0 1 --2- 10.20.193.27:0/210307225 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fcd3c112fc0 0x7fcd3c0f8260 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.078 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.036+0000 7fcd40a5b6c0 1 -- 10.20.193.27:0/210307225 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcd3c10c4e0 msgr2=0x7fcd3c0f2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.078 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.036+0000 7fcd40a5b6c0 1 --2- 10.20.193.27:0/210307225 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcd3c10c4e0 0x7fcd3c0f2960 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.078 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.036+0000 7fcd40a5b6c0 1 -- 10.20.193.27:0/210307225 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd2c0094c0 con 0x7fcd3c112b10 2026-02-13T02:22:27.079 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.036+0000 7efdd17fa6c0 1 --2- 10.20.193.27:0/922388294 >> [v2:10.20.193.107:6824/1496433214,v1:10.20.193.107:6825/1496433214] conn(0x7efda80dcd70 0x7efda80fd180 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.079 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.036+0000 7efdd17fa6c0 1 -- 10.20.193.27:0/922388294 --> [v2:10.20.193.107:6824/1496433214,v1:10.20.193.107:6825/1496433214] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efda80fd830 con 0x7efda80dcd70 2026-02-13T02:22:27.079 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.036+0000 7efdd17fa6c0 1 -- 10.20.193.27:0/922388294 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7efdc00e3d70 con 0x7efdd410f200 2026-02-13T02:22:27.079 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.036+0000 7fcd40a5b6c0 1 --2- 10.20.193.27:0/210307225 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcd3c112b10 0x7fcd3c0f2ea0 secure :-1 s=READY pgs=274 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcd3000b750 tx=0x7fcd3000bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.079 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.036+0000 7efdd37fe6c0 1 --2- 10.20.193.27:0/922388294 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7efda80b5790 0x7efda80d5c40 secure :-1 s=READY pgs=190 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7efdc4005980 tx=0x7efdc4005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.079 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.036+0000 7f2c89ffb6c0 1 -- 10.20.193.27:0/300191952 <== mon.1 v2:10.20.193.107:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2c7c007ea0 con 0x7f2c94112b10 2026-02-13T02:22:27.079 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.036+0000 7efdd8b4f6c0 1 --2- 10.20.193.27:0/922388294 >> [v2:10.20.193.107:6824/1496433214,v1:10.20.193.107:6825/1496433214] conn(0x7efda80dcd70 0x7efda80fd180 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.079 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.036+0000 7ff5667fc6c0 1 -- 10.20.193.27:0/3678529562 <== osd.10 v2:10.20.193.28:6824/2255755797 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33498 (crc 0 0 0) 0x7ff5540fd830 con 0x7ff5540dcd70 2026-02-13T02:22:27.080 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.036+0000 7fcd4125c6c0 1 --2- 10.20.193.27:0/210307225 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcd3c10c4e0 0x7fcd3c0f2960 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:22:27.080 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.036+0000 7fcd2a7fc6c0 1 -- 10.20.193.27:0/210307225 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fcd30004a80 con 0x7fcd3c112b10 2026-02-13T02:22:27.080 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.036+0000 7fcd2a7fc6c0 1 -- 10.20.193.27:0/210307225 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fcd30009030 con 0x7fcd3c112b10 2026-02-13T02:22:27.080 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.036+0000 7fcd2a7fc6c0 1 -- 10.20.193.27:0/210307225 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fcd3000ce20 con 0x7fcd3c112b10 2026-02-13T02:22:27.080 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.036+0000 7fcd434eb6c0 1 -- 10.20.193.27:0/210307225 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd3800a660 con 0x7fcd3c112b10 2026-02-13T02:22:27.080 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.036+0000 7fcd434eb6c0 1 -- 10.20.193.27:0/210307225 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd3c0f93d0 con 0x7fcd3c112b10 2026-02-13T02:22:27.080 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.036+0000 7efdd8b4f6c0 1 --2- 10.20.193.27:0/922388294 >> [v2:10.20.193.107:6824/1496433214,v1:10.20.193.107:6825/1496433214] conn(0x7efda80dcd70 0x7efda80fd180 crc :-1 s=READY pgs=6 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.080 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.037+0000 7ff57d1156c0 1 --2- 10.20.193.27:0/3678529562 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff5540b5790 0x7ff5540d5c40 secure :-1 s=READY pgs=191 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff568004770 tx=0x7ff568006c90 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.080 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.037+0000 7f2c9ae746c0 1 -- 10.20.193.27:0/300191952 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c9410a540 con 0x7f2c94112b10 2026-02-13T02:22:27.080 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.037+0000 7f2c9ae746c0 1 -- 10.20.193.27:0/300191952 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c9410a7c0 con 0x7f2c94112b10 2026-02-13T02:22:27.080 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.037+0000 7fb7517fa6c0 1 -- 10.20.193.27:0/2757875259 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7fb7400077f0 con 0x7fb75410f230 2026-02-13T02:22:27.081 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.037+0000 7efdd17fa6c0 1 -- 10.20.193.27:0/922388294 <== osd.11 v2:10.20.193.107:6824/1496433214 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33498 (crc 0 0 0) 0x7efda80fd830 con 0x7efda80dcd70 2026-02-13T02:22:27.081 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.037+0000 7f2c9ae746c0 1 -- 10.20.193.27:0/300191952 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2c94052a50 con 0x7f2c94112b10 2026-02-13T02:22:27.081 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.038+0000 7fb7517fa6c0 1 --2- 10.20.193.27:0/2757875259 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb7280b5710 0x7fb7280d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.081 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.038+0000 7fb7517fa6c0 1 -- 10.20.193.27:0/2757875259 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7fb740107e50 con 0x7fb75410f230 2026-02-13T02:22:27.081 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.038+0000 7fb7517fa6c0 1 --2- 10.20.193.27:0/2757875259 >> [v2:10.20.193.107:6808/4246100128,v1:10.20.193.107:6809/4246100128] conn(0x7fb7280dccf0 0x7fb7280fd100 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.081 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.038+0000 7fb7517fa6c0 1 -- 10.20.193.27:0/2757875259 --> [v2:10.20.193.107:6808/4246100128,v1:10.20.193.107:6809/4246100128] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb7280fd7b0 con 0x7fb7280dccf0 2026-02-13T02:22:27.081 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.038+0000 7fb7517fa6c0 1 -- 10.20.193.27:0/2757875259 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fb740108200 con 0x7fb75410f230 2026-02-13T02:22:27.081 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.038+0000 7fb7537fe6c0 1 --2- 10.20.193.27:0/2757875259 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb7280b5710 0x7fb7280d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.081 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.038+0000 7fb758f8e6c0 1 --2- 10.20.193.27:0/2757875259 >> [v2:10.20.193.107:6808/4246100128,v1:10.20.193.107:6809/4246100128] conn(0x7fb7280dccf0 0x7fb7280fd100 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.081 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.038+0000 7fb7537fe6c0 1 --2- 10.20.193.27:0/2757875259 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb7280b5710 0x7fb7280d5bc0 secure :-1 s=READY pgs=192 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb754123a30 tx=0x7fb744009260 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.081 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.038+0000 7fb758f8e6c0 1 --2- 10.20.193.27:0/2757875259 >> [v2:10.20.193.107:6808/4246100128,v1:10.20.193.107:6809/4246100128] conn(0x7fb7280dccf0 0x7fb7280fd100 crc :-1 s=READY pgs=8 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.081 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.038+0000 7f2c89ffb6c0 1 -- 10.20.193.27:0/300191952 <== mon.1 v2:10.20.193.107:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f2c7c0071c0 con 0x7f2c94112b10 2026-02-13T02:22:27.081 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.039+0000 7fb7517fa6c0 1 -- 10.20.193.27:0/2757875259 <== osd.5 v2:10.20.193.107:6808/4246100128 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33498 (crc 0 0 0) 0x7fb7280fd7b0 con 0x7fb7280dccf0 2026-02-13T02:22:27.081 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.038+0000 7fcd434eb6c0 1 -- 10.20.193.27:0/210307225 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcd3c052a50 con 0x7fcd3c112b10 2026-02-13T02:22:27.082 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.039+0000 7f2c89ffb6c0 1 --2- 10.20.193.27:0/300191952 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f2c700b5660 0x7f2c700d5b10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.082 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.039+0000 7f2c98be56c0 1 --2- 10.20.193.27:0/300191952 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f2c700b5660 0x7f2c700d5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.082 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.039+0000 7f2c89ffb6c0 1 -- 10.20.193.27:0/300191952 <== mon.1 v2:10.20.193.107:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f2c7c107bd0 con 0x7f2c94112b10 2026-02-13T02:22:27.082 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.039+0000 7f2c89ffb6c0 1 --2- 10.20.193.27:0/300191952 >> [v2:10.20.193.107:6816/1331562104,v1:10.20.193.107:6817/1331562104] conn(0x7f2c700dcad0 0x7f2c700fcee0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.082 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.039+0000 7f2c89ffb6c0 1 -- 10.20.193.27:0/300191952 --> [v2:10.20.193.107:6816/1331562104,v1:10.20.193.107:6817/1331562104] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2c700fd590 con 0x7f2c700dcad0 2026-02-13T02:22:27.082 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.039+0000 7f2c89ffb6c0 1 -- 10.20.193.27:0/300191952 <== mon.1 v2:10.20.193.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f2c7c107f80 con 0x7f2c94112b10 2026-02-13T02:22:27.082 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.039+0000 7f2c98be56c0 1 --2- 10.20.193.27:0/300191952 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f2c700b5660 0x7f2c700d5b10 secure :-1 s=READY pgs=193 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2c84004750 tx=0x7f2c84005f90 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.082 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.039+0000 7f2c993e66c0 1 --2- 10.20.193.27:0/300191952 >> [v2:10.20.193.107:6816/1331562104,v1:10.20.193.107:6817/1331562104] conn(0x7f2c700dcad0 0x7f2c700fcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.082 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.039+0000 7fcd2a7fc6c0 1 -- 10.20.193.27:0/210307225 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7fcd30005000 con 0x7fcd3c112b10 2026-02-13T02:22:27.082 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.040+0000 7fcd2a7fc6c0 1 --2- 10.20.193.27:0/210307225 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fcd180b5690 0x7fcd180d5b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.082 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.040+0000 7fcd2a7fc6c0 1 -- 10.20.193.27:0/210307225 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7fcd300e3a80 con 0x7fcd3c112b10 2026-02-13T02:22:27.082 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.040+0000 7f2c993e66c0 1 --2- 10.20.193.27:0/300191952 >> [v2:10.20.193.107:6816/1331562104,v1:10.20.193.107:6817/1331562104] conn(0x7f2c700dcad0 0x7f2c700fcee0 crc :-1 s=READY pgs=8 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.083 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.040+0000 7fcd2a7fc6c0 1 --2- 10.20.193.27:0/210307225 >> [v2:10.20.193.28:6816/1405200379,v1:10.20.193.28:6817/1405200379] conn(0x7fcd180dcc70 0x7fcd180fd080 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.083 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.040+0000 7fcd2a7fc6c0 1 -- 10.20.193.27:0/210307225 --> [v2:10.20.193.28:6816/1405200379,v1:10.20.193.28:6817/1405200379] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcd180fd730 con 0x7fcd180dcc70 2026-02-13T02:22:27.083 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.040+0000 7fcd2a7fc6c0 1 -- 10.20.193.27:0/210307225 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fcd300e7150 con 0x7fcd3c112b10 2026-02-13T02:22:27.083 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.040+0000 7fcd41a5d6c0 1 --2- 10.20.193.27:0/210307225 >> [v2:10.20.193.28:6816/1405200379,v1:10.20.193.28:6817/1405200379] conn(0x7fcd180dcc70 0x7fcd180fd080 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.083 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.040+0000 7fcd4125c6c0 1 --2- 10.20.193.27:0/210307225 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fcd180b5690 0x7fcd180d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.083 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.041+0000 7f2c89ffb6c0 1 -- 10.20.193.27:0/300191952 <== osd.8 v2:10.20.193.107:6816/1331562104 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33498 (crc 0 0 0) 0x7f2c700fd590 con 0x7f2c700dcad0 2026-02-13T02:22:27.083 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.041+0000 7fcd4125c6c0 1 --2- 10.20.193.27:0/210307225 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fcd180b5690 0x7fcd180d5b40 secure :-1 s=READY pgs=194 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcd2c004750 tx=0x7fcd2c009260 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.083 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.044+0000 7fcd41a5d6c0 1 --2- 10.20.193.27:0/210307225 >> [v2:10.20.193.28:6816/1405200379,v1:10.20.193.28:6817/1405200379] conn(0x7fcd180dcc70 0x7fcd180fd080 crc :-1 s=READY pgs=7 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.083 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.044+0000 7ff57f3a06c0 1 -- 10.20.193.27:0/3678529562 --> [v2:10.20.193.28:6824/2255755797,v1:10.20.193.28:6825/2255755797] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff578065290 con 0x7ff5540dcd70 2026-02-13T02:22:27.083 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.044+0000 7fcd2a7fc6c0 1 -- 10.20.193.27:0/210307225 <== osd.7 v2:10.20.193.28:6816/1405200379 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33498 (crc 0 0 0) 0x7fcd180fd730 con 0x7fcd180dcc70 2026-02-13T02:22:27.084 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.044+0000 7ff5667fc6c0 1 -- 10.20.193.27:0/3678529562 <== osd.10 v2:10.20.193.28:6824/2255755797 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff578065290 con 0x7ff5540dcd70 2026-02-13T02:22:27.084 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.045+0000 7ff57f3a06c0 1 -- 10.20.193.27:0/3678529562 >> [v2:10.20.193.28:6824/2255755797,v1:10.20.193.28:6825/2255755797] conn(0x7ff5540dcd70 msgr2=0x7ff5540fd180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.084 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.045+0000 7ff57f3a06c0 1 --2- 10.20.193.27:0/3678529562 >> [v2:10.20.193.28:6824/2255755797,v1:10.20.193.28:6825/2255755797] conn(0x7ff5540dcd70 0x7ff5540fd180 crc :-1 s=READY pgs=9 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.084 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.045+0000 7ff57f3a06c0 1 -- 10.20.193.27:0/3678529562 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff5540b5790 msgr2=0x7ff5540d5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.084 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.045+0000 7ff57f3a06c0 1 --2- 10.20.193.27:0/3678529562 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff5540b5790 0x7ff5540d5c40 secure :-1 s=READY pgs=191 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff568004770 tx=0x7ff568006c90 comp rx=0 tx=0).stop 2026-02-13T02:22:27.084 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.045+0000 7ff57f3a06c0 1 -- 10.20.193.27:0/3678529562 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff578115440 msgr2=0x7ff57811ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.084 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.045+0000 7ff57f3a06c0 1 --2- 10.20.193.27:0/3678529562 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff578115440 0x7ff57811ce50 secure :-1 s=READY pgs=271 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff56c0023c0 tx=0x7ff56c02fc00 comp rx=0 tx=0).stop 2026-02-13T02:22:27.084 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.045+0000 7ff57d9166c0 1 -- 10.20.193.27:0/3678529562 reap_dead start 2026-02-13T02:22:27.084 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.045+0000 7ff57f3a06c0 1 -- 10.20.193.27:0/3678529562 shutdown_connections 2026-02-13T02:22:27.084 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.045+0000 7ff57f3a06c0 1 -- 10.20.193.27:0/3678529562 >> 10.20.193.27:0/3678529562 conn(0x7ff5780ecb30 msgr2=0x7ff57810cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.084 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.045+0000 7ff57f3a06c0 1 -- 10.20.193.27:0/3678529562 shutdown_connections 2026-02-13T02:22:27.084 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.045+0000 7ff57f3a06c0 1 -- 10.20.193.27:0/3678529562 wait complete. 2026-02-13T02:22:27.084 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.046+0000 7efdda5dd6c0 1 -- 10.20.193.27:0/922388294 --> [v2:10.20.193.107:6824/1496433214,v1:10.20.193.107:6825/1496433214] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efdd41792f0 con 0x7efda80dcd70 2026-02-13T02:22:27.085 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.047+0000 7efdd17fa6c0 1 -- 10.20.193.27:0/922388294 <== osd.11 v2:10.20.193.107:6824/1496433214 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7efdd41792f0 con 0x7efda80dcd70 2026-02-13T02:22:27.085 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.047+0000 7efdda5dd6c0 1 -- 10.20.193.27:0/922388294 >> [v2:10.20.193.107:6824/1496433214,v1:10.20.193.107:6825/1496433214] conn(0x7efda80dcd70 msgr2=0x7efda80fd180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.085 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.047+0000 7efdda5dd6c0 1 --2- 10.20.193.27:0/922388294 >> [v2:10.20.193.107:6824/1496433214,v1:10.20.193.107:6825/1496433214] conn(0x7efda80dcd70 0x7efda80fd180 crc :-1 s=READY pgs=6 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.085 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.047+0000 7efdda5dd6c0 1 -- 10.20.193.27:0/922388294 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7efda80b5790 msgr2=0x7efda80d5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.085 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.047+0000 7efdda5dd6c0 1 --2- 10.20.193.27:0/922388294 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7efda80b5790 0x7efda80d5c40 secure :-1 s=READY pgs=190 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7efdc4005980 tx=0x7efdc4005910 comp rx=0 tx=0).stop 2026-02-13T02:22:27.085 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.047+0000 7efdda5dd6c0 1 -- 10.20.193.27:0/922388294 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7efdd410f200 msgr2=0x7efdd40f4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.085 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.047+0000 7efdda5dd6c0 1 --2- 10.20.193.27:0/922388294 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7efdd410f200 0x7efdd40f4af0 secure :-1 s=READY pgs=272 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7efdc0005dc0 tx=0x7efdc000bbb0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.085 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.047+0000 7efdd8b4f6c0 1 -- 10.20.193.27:0/922388294 reap_dead start 2026-02-13T02:22:27.085 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.048+0000 7efdda5dd6c0 1 -- 10.20.193.27:0/922388294 shutdown_connections 2026-02-13T02:22:27.085 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.048+0000 7efdda5dd6c0 1 -- 10.20.193.27:0/922388294 >> 10.20.193.27:0/922388294 conn(0x7efdd40ecb00 msgr2=0x7efdd410d720 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.085 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.048+0000 7fb75aa186c0 1 -- 10.20.193.27:0/2757875259 --> [v2:10.20.193.107:6808/4246100128,v1:10.20.193.107:6809/4246100128] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb754052a50 con 0x7fb7280dccf0 2026-02-13T02:22:27.085 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.048+0000 7fb7517fa6c0 1 -- 10.20.193.27:0/2757875259 <== osd.5 v2:10.20.193.107:6808/4246100128 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb754052a50 con 0x7fb7280dccf0 2026-02-13T02:22:27.086 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.048+0000 7efdda5dd6c0 1 -- 10.20.193.27:0/922388294 shutdown_connections 2026-02-13T02:22:27.086 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.048+0000 7efdda5dd6c0 1 -- 10.20.193.27:0/922388294 wait complete. 2026-02-13T02:22:27.086 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.048+0000 7fb75aa186c0 1 -- 10.20.193.27:0/2757875259 >> [v2:10.20.193.107:6808/4246100128,v1:10.20.193.107:6809/4246100128] conn(0x7fb7280dccf0 msgr2=0x7fb7280fd100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.086 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.048+0000 7fb75aa186c0 1 --2- 10.20.193.27:0/2757875259 >> [v2:10.20.193.107:6808/4246100128,v1:10.20.193.107:6809/4246100128] conn(0x7fb7280dccf0 0x7fb7280fd100 crc :-1 s=READY pgs=8 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.086 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.048+0000 7fb75aa186c0 1 -- 10.20.193.27:0/2757875259 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb7280b5710 msgr2=0x7fb7280d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.086 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.048+0000 7fb75aa186c0 1 --2- 10.20.193.27:0/2757875259 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb7280b5710 0x7fb7280d5bc0 secure :-1 s=READY pgs=192 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb754123a30 tx=0x7fb744009260 comp rx=0 tx=0).stop 2026-02-13T02:22:27.086 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.048+0000 7fb75aa186c0 1 -- 10.20.193.27:0/2757875259 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb75410f230 msgr2=0x7fb75411d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.086 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.048+0000 7fb75aa186c0 1 --2- 10.20.193.27:0/2757875259 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb75410f230 0x7fb75411d0a0 secure :-1 s=READY pgs=273 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb7400028e0 tx=0x7fb740002910 comp rx=0 tx=0).stop 2026-02-13T02:22:27.086 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.048+0000 7fb758f8e6c0 1 -- 10.20.193.27:0/2757875259 reap_dead start 2026-02-13T02:22:27.086 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.048+0000 7fb75aa186c0 1 -- 10.20.193.27:0/2757875259 shutdown_connections 2026-02-13T02:22:27.086 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.048+0000 7fb75aa186c0 1 -- 10.20.193.27:0/2757875259 >> 10.20.193.27:0/2757875259 conn(0x7fb7540ecb30 msgr2=0x7fb75410cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.086 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.048+0000 7fb75aa186c0 1 -- 10.20.193.27:0/2757875259 shutdown_connections 2026-02-13T02:22:27.086 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.048+0000 7fb75aa186c0 1 -- 10.20.193.27:0/2757875259 wait complete. 2026-02-13T02:22:27.086 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.049+0000 7f2c9ae746c0 1 -- 10.20.193.27:0/300191952 --> [v2:10.20.193.107:6816/1331562104,v1:10.20.193.107:6817/1331562104] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2c9410eea0 con 0x7f2c700dcad0 2026-02-13T02:22:27.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.049+0000 7f2c89ffb6c0 1 -- 10.20.193.27:0/300191952 <== osd.8 v2:10.20.193.107:6816/1331562104 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2c9410eea0 con 0x7f2c700dcad0 2026-02-13T02:22:27.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.049+0000 7f2c9ae746c0 1 -- 10.20.193.27:0/300191952 >> [v2:10.20.193.107:6816/1331562104,v1:10.20.193.107:6817/1331562104] conn(0x7f2c700dcad0 msgr2=0x7f2c700fcee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.049+0000 7f2c9ae746c0 1 --2- 10.20.193.27:0/300191952 >> [v2:10.20.193.107:6816/1331562104,v1:10.20.193.107:6817/1331562104] conn(0x7f2c700dcad0 0x7f2c700fcee0 crc :-1 s=READY pgs=8 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.049+0000 7f2c9ae746c0 1 -- 10.20.193.27:0/300191952 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f2c700b5660 msgr2=0x7f2c700d5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.049+0000 7f2c9ae746c0 1 --2- 10.20.193.27:0/300191952 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f2c700b5660 0x7f2c700d5b10 secure :-1 s=READY pgs=193 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2c84004750 tx=0x7f2c84005f90 comp rx=0 tx=0).stop 2026-02-13T02:22:27.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.049+0000 7f2c9ae746c0 1 -- 10.20.193.27:0/300191952 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2c94112b10 msgr2=0x7f2c94104580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.049+0000 7f2c9ae746c0 1 --2- 10.20.193.27:0/300191952 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2c94112b10 0x7f2c94104580 secure :-1 s=READY pgs=54 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2c7c0098c0 tx=0x7f2c7c02fcc0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.049+0000 7f2c993e66c0 1 -- 10.20.193.27:0/300191952 reap_dead start 2026-02-13T02:22:27.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.050+0000 7f2c9ae746c0 1 -- 10.20.193.27:0/300191952 shutdown_connections 2026-02-13T02:22:27.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.050+0000 7f2c9ae746c0 1 -- 10.20.193.27:0/300191952 >> 10.20.193.27:0/300191952 conn(0x7f2c94063af0 msgr2=0x7f2c9410e540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.050+0000 7f2c9ae746c0 1 -- 10.20.193.27:0/300191952 shutdown_connections 2026-02-13T02:22:27.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.050+0000 7f2c9ae746c0 1 -- 10.20.193.27:0/300191952 wait complete. 2026-02-13T02:22:27.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.052+0000 7fcd434eb6c0 1 -- 10.20.193.27:0/210307225 --> [v2:10.20.193.28:6816/1405200379,v1:10.20.193.28:6817/1405200379] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcd3c10fc00 con 0x7fcd180dcc70 2026-02-13T02:22:27.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.053+0000 7fcd2a7fc6c0 1 -- 10.20.193.27:0/210307225 <== osd.7 v2:10.20.193.28:6816/1405200379 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcd3c10fc00 con 0x7fcd180dcc70 2026-02-13T02:22:27.088 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.053+0000 7fcd434eb6c0 1 -- 10.20.193.27:0/210307225 >> [v2:10.20.193.28:6816/1405200379,v1:10.20.193.28:6817/1405200379] conn(0x7fcd180dcc70 msgr2=0x7fcd180fd080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.088 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.054+0000 7fcd434eb6c0 1 --2- 10.20.193.27:0/210307225 >> [v2:10.20.193.28:6816/1405200379,v1:10.20.193.28:6817/1405200379] conn(0x7fcd180dcc70 0x7fcd180fd080 crc :-1 s=READY pgs=7 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.088 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.054+0000 7fcd434eb6c0 1 -- 10.20.193.27:0/210307225 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fcd180b5690 msgr2=0x7fcd180d5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.088 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.054+0000 7fcd434eb6c0 1 --2- 10.20.193.27:0/210307225 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fcd180b5690 0x7fcd180d5b40 secure :-1 s=READY pgs=194 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcd2c004750 tx=0x7fcd2c009260 comp rx=0 tx=0).stop 2026-02-13T02:22:27.088 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.054+0000 7fcd434eb6c0 1 -- 10.20.193.27:0/210307225 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcd3c112b10 msgr2=0x7fcd3c0f2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.088 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.054+0000 7fcd434eb6c0 1 --2- 10.20.193.27:0/210307225 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcd3c112b10 0x7fcd3c0f2ea0 secure :-1 s=READY pgs=274 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcd3000b750 tx=0x7fcd3000bc20 comp rx=0 tx=0).stop 2026-02-13T02:22:27.088 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.054+0000 7fcd41a5d6c0 1 -- 10.20.193.27:0/210307225 reap_dead start 2026-02-13T02:22:27.088 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.054+0000 7fcd434eb6c0 1 -- 10.20.193.27:0/210307225 shutdown_connections 2026-02-13T02:22:27.088 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.054+0000 7fcd434eb6c0 1 -- 10.20.193.27:0/210307225 >> 10.20.193.27:0/210307225 conn(0x7fcd3c063af0 msgr2=0x7fcd3c10f220 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.088 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.054+0000 7fcd434eb6c0 1 -- 10.20.193.27:0/210307225 shutdown_connections 2026-02-13T02:22:27.088 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.054+0000 7fcd434eb6c0 1 -- 10.20.193.27:0/210307225 wait complete. 2026-02-13T02:22:27.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.074+0000 7fede15086c0 1 -- 10.20.193.27:0/2702554979 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7feddc1131c0 msgr2=0x7feddc173b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.074+0000 7fede15086c0 1 --2- 10.20.193.27:0/2702554979 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7feddc1131c0 0x7feddc173b60 secure :-1 s=READY pgs=275 gs=3 cs=0 l=1 c_cookie=4963149788760274938 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fedcc00a9b0 tx=0x7fedcc02fd20 comp rx=0 tx=0).stop 2026-02-13T02:22:27.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.075+0000 7fede15086c0 1 -- 10.20.193.27:0/2702554979 shutdown_connections 2026-02-13T02:22:27.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.075+0000 7fede15086c0 1 --2- 10.20.193.27:0/2702554979 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7feddc1131c0 0x7feddc173b60 unknown :-1 s=CLOSED pgs=275 gs=3 cs=0 l=1 c_cookie=4963149788760274938 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.075+0000 7fede15086c0 1 --2- 10.20.193.27:0/2702554979 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7feddc10d9a0 0x7feddc1129e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.075+0000 7fede15086c0 1 --2- 10.20.193.27:0/2702554979 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7feddc10cfb0 0x7feddc10d380 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.075+0000 7fede15086c0 1 -- 10.20.193.27:0/2702554979 >> 10.20.193.27:0/2702554979 conn(0x7feddc0ea8b0 msgr2=0x7feddc10acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.075+0000 7fede15086c0 1 -- 10.20.193.27:0/2702554979 shutdown_connections 2026-02-13T02:22:27.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.075+0000 7fede15086c0 1 -- 10.20.193.27:0/2702554979 wait complete. 2026-02-13T02:22:27.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.075+0000 7fede15086c0 1 Processor -- start 2026-02-13T02:22:27.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.075+0000 7fede15086c0 1 -- start start 2026-02-13T02:22:27.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.075+0000 7fede15086c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7feddc10cfb0 0x7feddc11af60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.075+0000 7fede15086c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7feddc1131c0 0x7feddc11b4a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.075+0000 7fede15086c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7feddc120860 0x7feddc120d00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.075+0000 7fede15086c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7feddc121270 con 0x7feddc10cfb0 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.075+0000 7fede15086c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7feddc1213e0 con 0x7feddc1131c0 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.075+0000 7fede15086c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7feddc1215a0 con 0x7feddc120860 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.075+0000 7fedda7fc6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7feddc1131c0 0x7feddc11b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.075+0000 7feddb7fe6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7feddc120860 0x7feddc120d00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.075+0000 7fedda7fc6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7feddc1131c0 0x7feddc11b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.27:47888/0 (socket says 10.20.193.27:47888) 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.075+0000 7fedda7fc6c0 1 -- 10.20.193.27:0/2316301093 learned_addr learned my addr 10.20.193.27:0/2316301093 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.076+0000 7feddaffd6c0 1 --2- 10.20.193.27:0/2316301093 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7feddc10cfb0 0x7feddc11af60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.076+0000 7fedda7fc6c0 1 -- 10.20.193.27:0/2316301093 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7feddc120860 msgr2=0x7feddc120d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.076+0000 7fedda7fc6c0 1 --2- 10.20.193.27:0/2316301093 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7feddc120860 0x7feddc120d00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.076+0000 7fedda7fc6c0 1 -- 10.20.193.27:0/2316301093 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7feddc10cfb0 msgr2=0x7feddc11af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.076+0000 7fedda7fc6c0 1 --2- 10.20.193.27:0/2316301093 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7feddc10cfb0 0x7feddc11af60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.076+0000 7fedda7fc6c0 1 -- 10.20.193.27:0/2316301093 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedcc00a5b0 con 0x7feddc1131c0 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.076+0000 7feddb7fe6c0 1 --2- 10.20.193.27:0/2316301093 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7feddc120860 0x7feddc120d00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.076+0000 7feddaffd6c0 1 --2- 10.20.193.27:0/2316301093 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7feddc10cfb0 0x7feddc11af60 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.076+0000 7fedda7fc6c0 1 --2- 10.20.193.27:0/2316301093 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7feddc1131c0 0x7feddc11b4a0 secure :-1 s=READY pgs=52 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fedc4005930 tx=0x7fedc400bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.076+0000 7fedbbfff6c0 1 -- 10.20.193.27:0/2316301093 <== mon.2 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fedc4004a80 con 0x7feddc1131c0 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.076+0000 7fedbbfff6c0 1 -- 10.20.193.27:0/2316301093 <== mon.2 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fedc4009030 con 0x7feddc1131c0 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.076+0000 7fedbbfff6c0 1 -- 10.20.193.27:0/2316301093 <== mon.2 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fedc400cd10 con 0x7feddc1131c0 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.076+0000 7fede15086c0 1 -- 10.20.193.27:0/2316301093 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feddc175c80 con 0x7feddc1131c0 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.077+0000 7fede15086c0 1 -- 10.20.193.27:0/2316301093 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feddc1760e0 con 0x7feddc1131c0 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.077+0000 7fede15086c0 1 -- 10.20.193.27:0/2316301093 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feddc052a50 con 0x7feddc1131c0 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.078+0000 7fedbbfff6c0 1 -- 10.20.193.27:0/2316301093 <== mon.2 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7fedc4004be0 con 0x7feddc1131c0 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.079+0000 7fedbbfff6c0 1 --2- 10.20.193.27:0/2316301093 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fedb00b5590 0x7fedb00d5a40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.079+0000 7fedbbfff6c0 1 -- 10.20.193.27:0/2316301093 <== mon.2 v2:10.20.193.28:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7fedc40e3c70 con 0x7feddc1131c0 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.079+0000 7feddaffd6c0 1 --2- 10.20.193.27:0/2316301093 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fedb00b5590 0x7fedb00d5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.079+0000 7feddaffd6c0 1 --2- 10.20.193.27:0/2316301093 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fedb00b5590 0x7fedb00d5a40 secure :-1 s=READY pgs=195 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fedd0004750 tx=0x7fedd0005f10 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.079+0000 7fedbbfff6c0 1 --2- 10.20.193.27:0/2316301093 >> [v2:10.20.193.27:6818/3514139178,v1:10.20.193.27:6819/3514139178] conn(0x7fedb00dca00 0x7fedb00fce10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.079+0000 7fedbbfff6c0 1 -- 10.20.193.27:0/2316301093 --> [v2:10.20.193.27:6818/3514139178,v1:10.20.193.27:6819/3514139178] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fedb00fd4c0 con 0x7fedb00dca00 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.079+0000 7fedbbfff6c0 1 -- 10.20.193.27:0/2316301093 <== mon.2 v2:10.20.193.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fedc40e4000 con 0x7feddc1131c0 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.080+0000 7feddb7fe6c0 1 --2- 10.20.193.27:0/2316301093 >> [v2:10.20.193.27:6818/3514139178,v1:10.20.193.27:6819/3514139178] conn(0x7fedb00dca00 0x7fedb00fce10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.080+0000 7feddb7fe6c0 1 --2- 10.20.193.27:0/2316301093 >> [v2:10.20.193.27:6818/3514139178,v1:10.20.193.27:6819/3514139178] conn(0x7fedb00dca00 0x7fedb00fce10 crc :-1 s=READY pgs=7 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.081+0000 7fedbbfff6c0 1 -- 10.20.193.27:0/2316301093 <== osd.6 v2:10.20.193.27:6818/3514139178 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33498 (crc 0 0 0) 0x7fedb00fd4c0 con 0x7fedb00dca00 2026-02-13T02:22:27.091 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.089+0000 7f9484aa86c0 1 -- 10.20.193.27:0/4093422021 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f948010fc20 msgr2=0x7f9480114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.091 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.089+0000 7f9484aa86c0 1 --2- 10.20.193.27:0/4093422021 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f948010fc20 0x7f9480114c60 secure :-1 s=READY pgs=276 gs=1 cs=0 l=1 c_cookie=16330323973726359240 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f946801adb0 tx=0x7f9468040110 comp rx=0 tx=0).stop 2026-02-13T02:22:27.091 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.089+0000 7f9484aa86c0 1 -- 10.20.193.27:0/4093422021 shutdown_connections 2026-02-13T02:22:27.091 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.089+0000 7f9484aa86c0 1 --2- 10.20.193.27:0/4093422021 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9480115440 0x7f9480175de0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.091 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.089+0000 7f9484aa86c0 1 --2- 10.20.193.27:0/4093422021 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f948010fc20 0x7f9480114c60 unknown :-1 s=CLOSED pgs=276 gs=1 cs=0 l=1 c_cookie=16330323973726359240 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.091 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.089+0000 7f9484aa86c0 1 --2- 10.20.193.27:0/4093422021 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f948010f230 0x7f948010f600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.091 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.089+0000 7f9484aa86c0 1 -- 10.20.193.27:0/4093422021 >> 10.20.193.27:0/4093422021 conn(0x7f94800ecb30 msgr2=0x7f948010cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.091 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.089+0000 7f9484aa86c0 1 -- 10.20.193.27:0/4093422021 shutdown_connections 2026-02-13T02:22:27.091 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.089+0000 7f9484aa86c0 1 -- 10.20.193.27:0/4093422021 wait complete. 2026-02-13T02:22:27.091 INFO:teuthology.orchestra.run.trial027.stdout:30064771096 2026-02-13T02:22:27.091 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.0 2026-02-13T02:22:27.092 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.089+0000 7fede15086c0 1 -- 10.20.193.27:0/2316301093 --> [v2:10.20.193.27:6818/3514139178,v1:10.20.193.27:6819/3514139178] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7feddc061a30 con 0x7fedb00dca00 2026-02-13T02:22:27.092 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.090+0000 7fedbbfff6c0 1 -- 10.20.193.27:0/2316301093 <== osd.6 v2:10.20.193.27:6818/3514139178 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7feddc061a30 con 0x7fedb00dca00 2026-02-13T02:22:27.092 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.090+0000 7f9484aa86c0 1 Processor -- start 2026-02-13T02:22:27.093 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.090+0000 7f9484aa86c0 1 -- start start 2026-02-13T02:22:27.093 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.090+0000 7f9484aa86c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f948010f230 0x7f9480066730 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.093 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.090+0000 7f9484aa86c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9480115440 0x7f9480062990 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.093 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.090+0000 7f9484aa86c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9480062f70 0x7f9480063410 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.093 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.090+0000 7f9484aa86c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9480162d70 con 0x7f948010f230 2026-02-13T02:22:27.093 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.090+0000 7f9484aa86c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f94801742f0 con 0x7f9480115440 2026-02-13T02:22:27.093 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.090+0000 7f9484aa86c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f94801744b0 con 0x7f9480062f70 2026-02-13T02:22:27.093 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.091+0000 7f947e5756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f948010f230 0x7f9480066730 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.093 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.091+0000 7f947e5756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f948010f230 0x7f9480066730 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55764/0 (socket says 10.20.193.27:55764) 2026-02-13T02:22:27.093 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.091+0000 7f947e5756c0 1 -- 10.20.193.27:0/2079330931 learned_addr learned my addr 10.20.193.27:0/2079330931 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:27.093 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.091+0000 7f947dd746c0 1 --2- 10.20.193.27:0/2079330931 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9480115440 0x7f9480062990 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.093 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.091+0000 7f947ed766c0 1 --2- 10.20.193.27:0/2079330931 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9480062f70 0x7f9480063410 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.093 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.091+0000 7f947e5756c0 1 -- 10.20.193.27:0/2079330931 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9480062f70 msgr2=0x7f9480063410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.093 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.091+0000 7f947e5756c0 1 --2- 10.20.193.27:0/2079330931 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9480062f70 0x7f9480063410 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.093 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.091+0000 7f947e5756c0 1 -- 10.20.193.27:0/2079330931 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9480115440 msgr2=0x7f9480062990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.093 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.091+0000 7f947e5756c0 1 --2- 10.20.193.27:0/2079330931 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9480115440 0x7f9480062990 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.093 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.091+0000 7f947e5756c0 1 -- 10.20.193.27:0/2079330931 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946801a9a0 con 0x7f948010f230 2026-02-13T02:22:27.093 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.091+0000 7f947ed766c0 1 --2- 10.20.193.27:0/2079330931 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9480062f70 0x7f9480063410 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:22:27.093 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.091+0000 7f947dd746c0 1 --2- 10.20.193.27:0/2079330931 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9480115440 0x7f9480062990 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:22:27.093 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.091+0000 7f947e5756c0 1 --2- 10.20.193.27:0/2079330931 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f948010f230 0x7f9480066730 secure :-1 s=READY pgs=277 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9474005dc0 tx=0x7f947400bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.093 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.091+0000 7fede15086c0 1 -- 10.20.193.27:0/2316301093 >> [v2:10.20.193.27:6818/3514139178,v1:10.20.193.27:6819/3514139178] conn(0x7fedb00dca00 msgr2=0x7fedb00fce10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.093 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.091+0000 7fede15086c0 1 --2- 10.20.193.27:0/2316301093 >> [v2:10.20.193.27:6818/3514139178,v1:10.20.193.27:6819/3514139178] conn(0x7fedb00dca00 0x7fedb00fce10 crc :-1 s=READY pgs=7 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.094 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.091+0000 7f94677fe6c0 1 -- 10.20.193.27:0/2079330931 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9474004250 con 0x7f948010f230 2026-02-13T02:22:27.094 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.091+0000 7f94677fe6c0 1 -- 10.20.193.27:0/2079330931 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9474009070 con 0x7f948010f230 2026-02-13T02:22:27.094 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.091+0000 7f94677fe6c0 1 -- 10.20.193.27:0/2079330931 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f947400cd70 con 0x7f948010f230 2026-02-13T02:22:27.094 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.091+0000 7fede15086c0 1 -- 10.20.193.27:0/2316301093 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fedb00b5590 msgr2=0x7fedb00d5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.094 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.091+0000 7fede15086c0 1 --2- 10.20.193.27:0/2316301093 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fedb00b5590 0x7fedb00d5a40 secure :-1 s=READY pgs=195 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fedd0004750 tx=0x7fedd0005f10 comp rx=0 tx=0).stop 2026-02-13T02:22:27.094 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.091+0000 7fede15086c0 1 -- 10.20.193.27:0/2316301093 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7feddc1131c0 msgr2=0x7feddc11b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.094 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.091+0000 7fede15086c0 1 --2- 10.20.193.27:0/2316301093 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7feddc1131c0 0x7feddc11b4a0 secure :-1 s=READY pgs=52 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fedc4005930 tx=0x7fedc400bc50 comp rx=0 tx=0).stop 2026-02-13T02:22:27.094 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.091+0000 7feddb7fe6c0 1 -- 10.20.193.27:0/2316301093 reap_dead start 2026-02-13T02:22:27.094 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.091+0000 7fede15086c0 1 -- 10.20.193.27:0/2316301093 shutdown_connections 2026-02-13T02:22:27.094 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.091+0000 7fede15086c0 1 -- 10.20.193.27:0/2316301093 >> 10.20.193.27:0/2316301093 conn(0x7feddc0ea8b0 msgr2=0x7feddc10ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.094 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.091+0000 7fede15086c0 1 -- 10.20.193.27:0/2316301093 shutdown_connections 2026-02-13T02:22:27.094 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.091+0000 7fede15086c0 1 -- 10.20.193.27:0/2316301093 wait complete. 2026-02-13T02:22:27.094 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.091+0000 7f9484aa86c0 1 -- 10.20.193.27:0/2079330931 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9480174790 con 0x7f948010f230 2026-02-13T02:22:27.094 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.091+0000 7f9484aa86c0 1 -- 10.20.193.27:0/2079330931 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9480174c20 con 0x7f948010f230 2026-02-13T02:22:27.094 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.091+0000 7f94657fa6c0 1 -- 10.20.193.27:0/2079330931 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9480052a50 con 0x7f948010f230 2026-02-13T02:22:27.095 INFO:teuthology.orchestra.run.trial027.stdout:111669149714 2026-02-13T02:22:27.095 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.4 2026-02-13T02:22:27.096 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.093+0000 7f94677fe6c0 1 -- 10.20.193.27:0/2079330931 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f94740028c0 con 0x7f948010f230 2026-02-13T02:22:27.096 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.093+0000 7f94677fe6c0 1 --2- 10.20.193.27:0/2079330931 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f94580b5690 0x7f94580d5b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.096 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.094+0000 7f94677fe6c0 1 -- 10.20.193.27:0/2079330931 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f94740e3b90 con 0x7f948010f230 2026-02-13T02:22:27.096 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.094+0000 7f947dd746c0 1 --2- 10.20.193.27:0/2079330931 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f94580b5690 0x7f94580d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.096 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.094+0000 7f94677fe6c0 1 --2- 10.20.193.27:0/2079330931 >> [v2:10.20.193.27:6826/1444927853,v1:10.20.193.27:6827/1444927853] conn(0x7f94580dcc70 0x7f94580fd080 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.096 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.094+0000 7f94677fe6c0 1 -- 10.20.193.27:0/2079330931 --> [v2:10.20.193.27:6826/1444927853,v1:10.20.193.27:6827/1444927853] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f94580fd730 con 0x7f94580dcc70 2026-02-13T02:22:27.096 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.094+0000 7f94677fe6c0 1 -- 10.20.193.27:0/2079330931 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f94740e3f40 con 0x7f948010f230 2026-02-13T02:22:27.096 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.094+0000 7f947ed766c0 1 --2- 10.20.193.27:0/2079330931 >> [v2:10.20.193.27:6826/1444927853,v1:10.20.193.27:6827/1444927853] conn(0x7f94580dcc70 0x7f94580fd080 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.096 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.094+0000 7f947dd746c0 1 --2- 10.20.193.27:0/2079330931 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f94580b5690 0x7f94580d5b40 secure :-1 s=READY pgs=196 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9468002fd0 tx=0x7f94680026b0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.096 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.094+0000 7f947ed766c0 1 --2- 10.20.193.27:0/2079330931 >> [v2:10.20.193.27:6826/1444927853,v1:10.20.193.27:6827/1444927853] conn(0x7f94580dcc70 0x7f94580fd080 crc :-1 s=READY pgs=8 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.096 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.094+0000 7f94677fe6c0 1 -- 10.20.193.27:0/2079330931 <== osd.9 v2:10.20.193.27:6826/1444927853 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33498 (crc 0 0 0) 0x7f94580fd730 con 0x7f94580dcc70 2026-02-13T02:22:27.098 INFO:teuthology.orchestra.run.trial027.stdout:55834574869 2026-02-13T02:22:27.099 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.2 2026-02-13T02:22:27.103 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.102+0000 7f94657fa6c0 1 -- 10.20.193.27:0/2079330931 --> [v2:10.20.193.27:6826/1444927853,v1:10.20.193.27:6827/1444927853] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9480058c80 con 0x7f94580dcc70 2026-02-13T02:22:27.103 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.102+0000 7f94677fe6c0 1 -- 10.20.193.27:0/2079330931 <== osd.9 v2:10.20.193.27:6826/1444927853 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9480058c80 con 0x7f94580dcc70 2026-02-13T02:22:27.104 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.102+0000 7f94657fa6c0 1 -- 10.20.193.27:0/2079330931 >> [v2:10.20.193.27:6826/1444927853,v1:10.20.193.27:6827/1444927853] conn(0x7f94580dcc70 msgr2=0x7f94580fd080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.104 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.102+0000 7f94657fa6c0 1 --2- 10.20.193.27:0/2079330931 >> [v2:10.20.193.27:6826/1444927853,v1:10.20.193.27:6827/1444927853] conn(0x7f94580dcc70 0x7f94580fd080 crc :-1 s=READY pgs=8 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.104 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.103+0000 7f94657fa6c0 1 -- 10.20.193.27:0/2079330931 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f94580b5690 msgr2=0x7f94580d5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.104 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.103+0000 7f94657fa6c0 1 --2- 10.20.193.27:0/2079330931 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f94580b5690 0x7f94580d5b40 secure :-1 s=READY pgs=196 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9468002fd0 tx=0x7f94680026b0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.104 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.103+0000 7f94657fa6c0 1 -- 10.20.193.27:0/2079330931 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f948010f230 msgr2=0x7f9480066730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.104 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.103+0000 7f94657fa6c0 1 --2- 10.20.193.27:0/2079330931 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f948010f230 0x7f9480066730 secure :-1 s=READY pgs=277 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9474005dc0 tx=0x7f947400bbb0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.104 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.103+0000 7f947ed766c0 1 -- 10.20.193.27:0/2079330931 reap_dead start 2026-02-13T02:22:27.105 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.103+0000 7f94657fa6c0 1 -- 10.20.193.27:0/2079330931 shutdown_connections 2026-02-13T02:22:27.105 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.103+0000 7f94657fa6c0 1 -- 10.20.193.27:0/2079330931 >> 10.20.193.27:0/2079330931 conn(0x7f94800ecb30 msgr2=0x7f948010cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.105 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.103+0000 7f94657fa6c0 1 -- 10.20.193.27:0/2079330931 shutdown_connections 2026-02-13T02:22:27.105 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.103+0000 7f94657fa6c0 1 -- 10.20.193.27:0/2079330931 wait complete. 2026-02-13T02:22:27.131 INFO:teuthology.orchestra.run.trial027.stdout:279172874247 2026-02-13T02:22:27.132 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.11 2026-02-13T02:22:27.141 INFO:teuthology.orchestra.run.trial027.stdout:201863462924 2026-02-13T02:22:27.142 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.8 2026-02-13T02:22:27.142 INFO:teuthology.orchestra.run.trial027.stdout:81604378644 2026-02-13T02:22:27.143 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.3 2026-02-13T02:22:27.149 INFO:teuthology.orchestra.run.trial027.stdout:171798691853 2026-02-13T02:22:27.149 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.7 2026-02-13T02:22:27.155 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:27 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3678262786' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-13T02:22:27.155 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:27 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3132368379' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-13T02:22:27.156 INFO:teuthology.orchestra.run.trial027.stdout:128849018897 2026-02-13T02:22:27.157 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.5 2026-02-13T02:22:27.158 INFO:teuthology.orchestra.run.trial027.stdout:231928233995 2026-02-13T02:22:27.158 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.9 2026-02-13T02:22:27.225 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:27.227 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:27.230 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:27.241 INFO:teuthology.orchestra.run.trial027.stdout:42949672983 2026-02-13T02:22:27.242 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.1 2026-02-13T02:22:27.273 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:27.275 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:27.280 INFO:teuthology.orchestra.run.trial027.stdout:158913789967 2026-02-13T02:22:27.281 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.6 2026-02-13T02:22:27.282 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:27.284 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:27.335 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:27.339 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:27.369 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.367+0000 7fb0029f46c0 1 -- 10.20.193.27:0/382078509 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7faffc10d9a0 msgr2=0x7faffc1129e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.369 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.367+0000 7fb0029f46c0 1 --2- 10.20.193.27:0/382078509 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7faffc10d9a0 0x7faffc1129e0 secure :-1 s=READY pgs=278 gs=1 cs=0 l=1 c_cookie=15046144803474212922 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fafec0099a0 tx=0x7fafec02ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:27.369 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.367+0000 7fb0029f46c0 1 -- 10.20.193.27:0/382078509 shutdown_connections 2026-02-13T02:22:27.369 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.367+0000 7fb0029f46c0 1 --2- 10.20.193.27:0/382078509 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7faffc1131c0 0x7faffc173b60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.369 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.367+0000 7fb0029f46c0 1 --2- 10.20.193.27:0/382078509 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7faffc10d9a0 0x7faffc1129e0 unknown :-1 s=CLOSED pgs=278 gs=1 cs=0 l=1 c_cookie=15046144803474212922 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.369 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.367+0000 7fb0029f46c0 1 --2- 10.20.193.27:0/382078509 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faffc10cfb0 0x7faffc10d380 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.369 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.367+0000 7fb0029f46c0 1 -- 10.20.193.27:0/382078509 >> 10.20.193.27:0/382078509 conn(0x7faffc0ea8b0 msgr2=0x7faffc10acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.370 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.367+0000 7fb0029f46c0 1 -- 10.20.193.27:0/382078509 shutdown_connections 2026-02-13T02:22:27.370 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.367+0000 7fb0029f46c0 1 -- 10.20.193.27:0/382078509 wait complete. 2026-02-13T02:22:27.370 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.367+0000 7fefc545a6c0 1 -- 10.20.193.27:0/1511685220 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fefb002fd70 con 0x7fefc010f200 2026-02-13T02:22:27.370 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.367+0000 7fefc86eb6c0 1 -- 10.20.193.27:0/1511685220 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fefc010f200 msgr2=0x7fefc010f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.371 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.367+0000 7fefc86eb6c0 1 --2- 10.20.193.27:0/1511685220 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fefc010f200 0x7fefc010f5d0 secure :-1 s=READY pgs=279 gs=1 cs=0 l=1 c_cookie=15081031764385504424 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fefb00099a0 tx=0x7fefb002ed70 comp rx=0 tx=0).stop 2026-02-13T02:22:27.371 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.367+0000 7fb0029f46c0 1 Processor -- start 2026-02-13T02:22:27.371 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.367+0000 7fefc86eb6c0 1 -- 10.20.193.27:0/1511685220 shutdown_connections 2026-02-13T02:22:27.371 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.367+0000 7fefc86eb6c0 1 --2- 10.20.193.27:0/1511685220 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fefc0115410 0x7fefc0175db0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.372 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.367+0000 7fefc86eb6c0 1 --2- 10.20.193.27:0/1511685220 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fefc010fbf0 0x7fefc0114c30 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.372 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.367+0000 7fefc86eb6c0 1 --2- 10.20.193.27:0/1511685220 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fefc010f200 0x7fefc010f5d0 unknown :-1 s=CLOSED pgs=279 gs=1 cs=0 l=1 c_cookie=15081031764385504424 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.372 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.367+0000 7fefc86eb6c0 1 -- 10.20.193.27:0/1511685220 >> 10.20.193.27:0/1511685220 conn(0x7fefc00ecb00 msgr2=0x7fefc010cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.372 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.368+0000 7fb0029f46c0 1 -- start start 2026-02-13T02:22:27.372 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.368+0000 7fefc86eb6c0 1 -- 10.20.193.27:0/1511685220 shutdown_connections 2026-02-13T02:22:27.373 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.368+0000 7fb0029f46c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faffc10cfb0 0x7faffc0f2aa0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.373 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.368+0000 7fb0029f46c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7faffc1131c0 0x7faffc173a60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.373 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.368+0000 7fb0029f46c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7faffc0f30d0 0x7faffc0f8350 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.373 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.368+0000 7fb0029f46c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7faffc0f37c0 con 0x7faffc1131c0 2026-02-13T02:22:27.373 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.368+0000 7fb0029f46c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7faffc0f8890 con 0x7faffc10cfb0 2026-02-13T02:22:27.373 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.368+0000 7fb0029f46c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7faffc0f8a50 con 0x7faffc0f30d0 2026-02-13T02:22:27.374 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.368+0000 7fefc86eb6c0 1 -- 10.20.193.27:0/1511685220 wait complete. 2026-02-13T02:22:27.374 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.368+0000 7fb000f6a6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7faffc0f30d0 0x7faffc0f8350 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.375 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.368+0000 7fb000f6a6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7faffc0f30d0 0x7faffc0f8350 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.107:3300/0 says I am v2:10.20.193.27:38276/0 (socket says 10.20.193.27:38276) 2026-02-13T02:22:27.375 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.368+0000 7fb000f6a6c0 1 -- 10.20.193.27:0/779774875 learned_addr learned my addr 10.20.193.27:0/779774875 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:27.375 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.368+0000 7faffb7fe6c0 1 --2- 10.20.193.27:0/779774875 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7faffc1131c0 0x7faffc173a60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.375 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.368+0000 7faffbfff6c0 1 --2- 10.20.193.27:0/779774875 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faffc10cfb0 0x7faffc0f2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.375 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.368+0000 7fb000f6a6c0 1 -- 10.20.193.27:0/779774875 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faffc10cfb0 msgr2=0x7faffc0f2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.375 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.368+0000 7fb000f6a6c0 1 --2- 10.20.193.27:0/779774875 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faffc10cfb0 0x7faffc0f2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.375 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.368+0000 7fb000f6a6c0 1 -- 10.20.193.27:0/779774875 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7faffc1131c0 msgr2=0x7faffc173a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.375 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.368+0000 7fb000f6a6c0 1 --2- 10.20.193.27:0/779774875 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7faffc1131c0 0x7faffc173a60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.375 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.368+0000 7fb000f6a6c0 1 -- 10.20.193.27:0/779774875 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafec009590 con 0x7faffc0f30d0 2026-02-13T02:22:27.375 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.368+0000 7faffb7fe6c0 1 --2- 10.20.193.27:0/779774875 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7faffc1131c0 0x7faffc173a60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:22:27.376 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.368+0000 7fefc86eb6c0 1 Processor -- start 2026-02-13T02:22:27.376 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.368+0000 7f3f723516c0 1 -- 10.20.193.27:0/2292894098 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3f6c061050 msgr2=0x7f3f6c155180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.376 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.368+0000 7f3f723516c0 1 --2- 10.20.193.27:0/2292894098 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3f6c061050 0x7f3f6c155180 secure :-1 s=READY pgs=280 gs=1 cs=0 l=1 c_cookie=11797904099709392404 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3f5c009970 tx=0x7f3f5c02ed50 comp rx=0 tx=0).stop 2026-02-13T02:22:27.376 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.368+0000 7f3f723516c0 1 -- 10.20.193.27:0/2292894098 shutdown_connections 2026-02-13T02:22:27.376 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.368+0000 7f3f723516c0 1 --2- 10.20.193.27:0/2292894098 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3f6c1556c0 0x7f3f6c175aa0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.377 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.368+0000 7f3f723516c0 1 --2- 10.20.193.27:0/2292894098 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3f6c061050 0x7f3f6c155180 unknown :-1 s=CLOSED pgs=280 gs=1 cs=0 l=1 c_cookie=11797904099709392404 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.377 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.368+0000 7f3f723516c0 1 --2- 10.20.193.27:0/2292894098 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3f6c060740 0x7f3f6c060b10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.377 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.368+0000 7f3f723516c0 1 -- 10.20.193.27:0/2292894098 >> 10.20.193.27:0/2292894098 conn(0x7f3f6c0ea8b0 msgr2=0x7f3f6c10acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.377 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.368+0000 7fefc86eb6c0 1 -- start start 2026-02-13T02:22:27.377 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.368+0000 7f3f723516c0 1 -- 10.20.193.27:0/2292894098 shutdown_connections 2026-02-13T02:22:27.378 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7f3f723516c0 1 -- 10.20.193.27:0/2292894098 wait complete. 2026-02-13T02:22:27.378 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7fb000f6a6c0 1 --2- 10.20.193.27:0/779774875 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7faffc0f30d0 0x7faffc0f8350 secure :-1 s=READY pgs=55 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7faff000c800 tx=0x7faff000ccd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.378 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7fefc86eb6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fefc010f200 0x7fefc011d230 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.379 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7fefc86eb6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fefc0115410 0x7fefc011d770 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.379 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7fefc86eb6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fefc0122b30 0x7fefc0122fd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.379 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7fefc86eb6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fefc0123540 con 0x7fefc0115410 2026-02-13T02:22:27.379 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7fefc86eb6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fefc01236b0 con 0x7fefc010f200 2026-02-13T02:22:27.379 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7fefc86eb6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fefc0123870 con 0x7fefc0122b30 2026-02-13T02:22:27.379 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7fefc5c5b6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fefc0115410 0x7fefc011d770 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.379 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7fefc5c5b6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fefc0115410 0x7fefc011d770 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:53454/0 (socket says 10.20.193.27:53454) 2026-02-13T02:22:27.379 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7fefc5c5b6c0 1 -- 10.20.193.27:0/2538192014 learned_addr learned my addr 10.20.193.27:0/2538192014 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:27.379 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7f3f723516c0 1 Processor -- start 2026-02-13T02:22:27.380 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7faff97fa6c0 1 -- 10.20.193.27:0/779774875 <== mon.1 v2:10.20.193.107:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7faff0003a30 con 0x7faffc0f30d0 2026-02-13T02:22:27.380 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7faff97fa6c0 1 -- 10.20.193.27:0/779774875 <== mon.1 v2:10.20.193.107:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7faff00054f0 con 0x7faffc0f30d0 2026-02-13T02:22:27.380 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7fb0029f46c0 1 -- 10.20.193.27:0/779774875 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faffc0f8d30 con 0x7faffc0f30d0 2026-02-13T02:22:27.380 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7faff97fa6c0 1 -- 10.20.193.27:0/779774875 <== mon.1 v2:10.20.193.107:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7faff000dc30 con 0x7faffc0f30d0 2026-02-13T02:22:27.381 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7f3f723516c0 1 -- start start 2026-02-13T02:22:27.381 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7f3f723516c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3f6c060740 0x7f3f6c1614a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.381 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7f3f723516c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3f6c061050 0x7f3f6c1619e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.381 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7f3f723516c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3f6c1556c0 0x7f3f6c166da0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.382 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7fefc6c5d6c0 1 --2- 10.20.193.27:0/2538192014 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fefc0122b30 0x7fefc0122fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.382 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7fefc5c5b6c0 1 -- 10.20.193.27:0/2538192014 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fefc0122b30 msgr2=0x7fefc0122fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.382 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7fefc5c5b6c0 1 --2- 10.20.193.27:0/2538192014 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fefc0122b30 0x7fefc0122fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.382 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7fefc5c5b6c0 1 -- 10.20.193.27:0/2538192014 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fefc010f200 msgr2=0x7fefc011d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.382 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7fefc5c5b6c0 1 --2- 10.20.193.27:0/2538192014 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fefc010f200 0x7fefc011d230 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.382 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7fefc5c5b6c0 1 -- 10.20.193.27:0/2538192014 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb0009590 con 0x7fefc0115410 2026-02-13T02:22:27.383 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7f3f723516c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6c1673f0 con 0x7f3f6c1556c0 2026-02-13T02:22:27.383 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7f3f723516c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6c167560 con 0x7f3f6c061050 2026-02-13T02:22:27.383 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7f3f723516c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6c167720 con 0x7f3f6c060740 2026-02-13T02:22:27.383 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7f3f708c36c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3f6c1556c0 0x7f3f6c166da0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.383 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7f3f708c36c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3f6c1556c0 0x7f3f6c166da0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:53464/0 (socket says 10.20.193.27:53464) 2026-02-13T02:22:27.383 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7fb0029f46c0 1 -- 10.20.193.27:0/779774875 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faffc0f94c0 con 0x7faffc0f30d0 2026-02-13T02:22:27.385 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7f3f708c36c0 1 -- 10.20.193.27:0/787528539 learned_addr learned my addr 10.20.193.27:0/787528539 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:27.385 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7f3f6b7fe6c0 1 --2- 10.20.193.27:0/787528539 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3f6c061050 0x7f3f6c1619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.385 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7f3f708c36c0 1 -- 10.20.193.27:0/787528539 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3f6c060740 msgr2=0x7f3f6c1614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.385 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7f3f708c36c0 1 --2- 10.20.193.27:0/787528539 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3f6c060740 0x7f3f6c1614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.385 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7f3f708c36c0 1 -- 10.20.193.27:0/787528539 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3f6c061050 msgr2=0x7f3f6c1619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.385 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7f3f708c36c0 1 --2- 10.20.193.27:0/787528539 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3f6c061050 0x7f3f6c1619e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.385 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7f3f708c36c0 1 -- 10.20.193.27:0/787528539 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f5c009590 con 0x7f3f6c1556c0 2026-02-13T02:22:27.385 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.370+0000 7f3f708c36c0 1 --2- 10.20.193.27:0/787528539 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3f6c1556c0 0x7f3f6c166da0 secure :-1 s=READY pgs=282 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3f6000c6c0 tx=0x7f3f6000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.385 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.370+0000 7f3f697fa6c0 1 -- 10.20.193.27:0/787528539 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3f600054a0 con 0x7f3f6c1556c0 2026-02-13T02:22:27.386 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7fefc5c5b6c0 1 --2- 10.20.193.27:0/2538192014 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fefc0115410 0x7fefc011d770 secure :-1 s=READY pgs=281 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fefb4009e00 tx=0x7fefb400ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.386 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.369+0000 7fefa77fe6c0 1 -- 10.20.193.27:0/2538192014 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fefb4004290 con 0x7fefc0115410 2026-02-13T02:22:27.386 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.370+0000 7fefc86eb6c0 1 -- 10.20.193.27:0/2538192014 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc00628d0 con 0x7fefc0115410 2026-02-13T02:22:27.386 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.370+0000 7fefa77fe6c0 1 -- 10.20.193.27:0/2538192014 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fefb4014e60 con 0x7fefc0115410 2026-02-13T02:22:27.386 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.370+0000 7fefc86eb6c0 1 -- 10.20.193.27:0/2538192014 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc0062e30 con 0x7fefc0115410 2026-02-13T02:22:27.387 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.370+0000 7f3f697fa6c0 1 -- 10.20.193.27:0/787528539 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3f6000a030 con 0x7f3f6c1556c0 2026-02-13T02:22:27.387 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.370+0000 7f3f723516c0 1 -- 10.20.193.27:0/787528539 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f6c167a00 con 0x7f3f6c1556c0 2026-02-13T02:22:27.387 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.370+0000 7fb0029f46c0 1 -- 10.20.193.27:0/779774875 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faffc0f9210 con 0x7faffc0f30d0 2026-02-13T02:22:27.387 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.370+0000 7fefa77fe6c0 1 -- 10.20.193.27:0/2538192014 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fefb4004290 con 0x7fefc0115410 2026-02-13T02:22:27.387 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.370+0000 7fefc86eb6c0 1 -- 10.20.193.27:0/2538192014 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc01104d0 con 0x7fefc0115410 2026-02-13T02:22:27.388 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.370+0000 7faff97fa6c0 1 -- 10.20.193.27:0/779774875 <== mon.1 v2:10.20.193.107:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7faff0005030 con 0x7faffc0f30d0 2026-02-13T02:22:27.388 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.371+0000 7f3f697fa6c0 1 -- 10.20.193.27:0/787528539 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3f6000dbf0 con 0x7f3f6c1556c0 2026-02-13T02:22:27.388 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.371+0000 7f3f723516c0 1 -- 10.20.193.27:0/787528539 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f6c14fdc0 con 0x7f3f6c1556c0 2026-02-13T02:22:27.389 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.371+0000 7faff97fa6c0 1 --2- 10.20.193.27:0/779774875 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fafd00b57e0 0x7fafd00d5c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.389 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.371+0000 7faff97fa6c0 1 -- 10.20.193.27:0/779774875 <== mon.1 v2:10.20.193.107:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7faff00e4aa0 con 0x7faffc0f30d0 2026-02-13T02:22:27.389 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.371+0000 7faffbfff6c0 1 --2- 10.20.193.27:0/779774875 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fafd00b57e0 0x7fafd00d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.389 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.371+0000 7f3f723516c0 1 -- 10.20.193.27:0/787528539 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f6c167c20 con 0x7f3f6c1556c0 2026-02-13T02:22:27.390 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.372+0000 7faffbfff6c0 1 --2- 10.20.193.27:0/779774875 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fafd00b57e0 0x7fafd00d5c90 secure :-1 s=READY pgs=197 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fafe8005980 tx=0x7fafe8005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.390 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.373+0000 7fefa77fe6c0 1 -- 10.20.193.27:0/2538192014 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7fefb40043f0 con 0x7fefc0115410 2026-02-13T02:22:27.390 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.373+0000 7fefa77fe6c0 1 --2- 10.20.193.27:0/2538192014 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fef9c0b56f0 0x7fef9c0d5ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.391 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.373+0000 7faff97fa6c0 1 -- 10.20.193.27:0/779774875 <== mon.1 v2:10.20.193.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7faff00a74c0 con 0x7faffc0f30d0 2026-02-13T02:22:27.391 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.374+0000 7fefc645c6c0 1 --2- 10.20.193.27:0/2538192014 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fef9c0b56f0 0x7fef9c0d5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.391 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.374+0000 7fefa77fe6c0 1 -- 10.20.193.27:0/2538192014 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7fefb40b2990 con 0x7fefc0115410 2026-02-13T02:22:27.391 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.374+0000 7f3f697fa6c0 1 -- 10.20.193.27:0/787528539 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f3f60003810 con 0x7f3f6c1556c0 2026-02-13T02:22:27.392 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.374+0000 7fefa77fe6c0 1 -- 10.20.193.27:0/2538192014 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fefb40ae780 con 0x7fefc0115410 2026-02-13T02:22:27.392 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.374+0000 7fefc645c6c0 1 --2- 10.20.193.27:0/2538192014 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fef9c0b56f0 0x7fef9c0d5ba0 secure :-1 s=READY pgs=198 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fefb00098c0 tx=0x7fefb0005c20 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.393 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.375+0000 7f3f697fa6c0 1 --2- 10.20.193.27:0/787528539 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3f400b5690 0x7f3f400d5b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.393 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.375+0000 7f3f697fa6c0 1 -- 10.20.193.27:0/787528539 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f3f600e49d0 con 0x7f3f6c1556c0 2026-02-13T02:22:27.393 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.375+0000 7f3f6bfff6c0 1 --2- 10.20.193.27:0/787528539 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3f400b5690 0x7f3f400d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.393 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.375+0000 7f3f6bfff6c0 1 --2- 10.20.193.27:0/787528539 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3f400b5690 0x7f3f400d5b40 secure :-1 s=READY pgs=199 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3f58004750 tx=0x7f3f58006d50 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.393 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.376+0000 7f3f697fa6c0 1 -- 10.20.193.27:0/787528539 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f3f600a73f0 con 0x7f3f6c1556c0 2026-02-13T02:22:27.408 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:27.411 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.409+0000 7f1e06c986c0 1 -- 10.20.193.27:0/1297818188 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1e000669a0 msgr2=0x7f1e000629b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.411 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.409+0000 7f1e06c986c0 1 --2- 10.20.193.27:0/1297818188 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1e000669a0 0x7f1e000629b0 secure :-1 s=READY pgs=56 gs=2 cs=0 l=1 c_cookie=7847989722811010767 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1df00098a0 tx=0x7f1df002ec40 comp rx=0 tx=0).stop 2026-02-13T02:22:27.411 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.409+0000 7f1e06c986c0 1 -- 10.20.193.27:0/1297818188 shutdown_connections 2026-02-13T02:22:27.411 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.409+0000 7f1e06c986c0 1 --2- 10.20.193.27:0/1297818188 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1e00159a50 0x7f1e00179eb0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.411 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.409+0000 7f1e06c986c0 1 --2- 10.20.193.27:0/1297818188 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1e00062fd0 0x7f1e00063400 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.412 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.409+0000 7f1e06c986c0 1 --2- 10.20.193.27:0/1297818188 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1e000669a0 0x7f1e000629b0 unknown :-1 s=CLOSED pgs=56 gs=2 cs=0 l=1 c_cookie=7847989722811010767 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.412 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.409+0000 7f1e06c986c0 1 -- 10.20.193.27:0/1297818188 >> 10.20.193.27:0/1297818188 conn(0x7f1e000ecb00 msgr2=0x7f1e0010cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.412 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.409+0000 7f1e06c986c0 1 -- 10.20.193.27:0/1297818188 shutdown_connections 2026-02-13T02:22:27.412 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.409+0000 7f1e06c986c0 1 -- 10.20.193.27:0/1297818188 wait complete. 2026-02-13T02:22:27.412 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.410+0000 7f1e06c986c0 1 Processor -- start 2026-02-13T02:22:27.412 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.410+0000 7f1e06c986c0 1 -- start start 2026-02-13T02:22:27.413 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.410+0000 7f1e06c986c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1e00062fd0 0x7f1e00163440 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.413 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.410+0000 7f1e06c986c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1e00159a50 0x7f1e00163980 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.414 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.410+0000 7f1e06c986c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1e00168d40 0x7f1e001691e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.414 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.410+0000 7f1e06c986c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1e00169750 con 0x7f1e00159a50 2026-02-13T02:22:27.414 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.410+0000 7f1e06c986c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1e001698c0 con 0x7f1e00062fd0 2026-02-13T02:22:27.414 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.410+0000 7f1e06c986c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1e00169a80 con 0x7f1e00168d40 2026-02-13T02:22:27.414 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.410+0000 7f1df7fff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1e00159a50 0x7f1e00163980 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.414 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.410+0000 7f1df7fff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1e00159a50 0x7f1e00163980 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:53486/0 (socket says 10.20.193.27:53486) 2026-02-13T02:22:27.414 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.410+0000 7f1df7fff6c0 1 -- 10.20.193.27:0/483797881 learned_addr learned my addr 10.20.193.27:0/483797881 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:27.414 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.410+0000 7f1e04a096c0 1 --2- 10.20.193.27:0/483797881 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1e00062fd0 0x7f1e00163440 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.414 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.410+0000 7f1e0520a6c0 1 --2- 10.20.193.27:0/483797881 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1e00168d40 0x7f1e001691e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.415 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.410+0000 7f5425a4d6c0 1 -- 10.20.193.27:0/3547258024 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5420115440 msgr2=0x7f5420175de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.415 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.410+0000 7f5425a4d6c0 1 --2- 10.20.193.27:0/3547258024 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5420115440 0x7f5420175de0 secure :-1 s=READY pgs=283 gs=1 cs=0 l=1 c_cookie=14810445126730600300 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f541000aa70 tx=0x7f541002fde0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.415 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.410+0000 7f5425a4d6c0 1 -- 10.20.193.27:0/3547258024 shutdown_connections 2026-02-13T02:22:27.415 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.410+0000 7f5425a4d6c0 1 --2- 10.20.193.27:0/3547258024 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5420115440 0x7f5420175de0 unknown :-1 s=CLOSED pgs=283 gs=1 cs=0 l=1 c_cookie=14810445126730600300 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.416 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.410+0000 7f5425a4d6c0 1 --2- 10.20.193.27:0/3547258024 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f542010fc20 0x7f5420114c60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.416 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.410+0000 7f5425a4d6c0 1 --2- 10.20.193.27:0/3547258024 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f542010f230 0x7f542010f600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.416 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.410+0000 7f5425a4d6c0 1 -- 10.20.193.27:0/3547258024 >> 10.20.193.27:0/3547258024 conn(0x7f54200ecb30 msgr2=0x7f542010cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.416 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.410+0000 7f5425a4d6c0 1 -- 10.20.193.27:0/3547258024 shutdown_connections 2026-02-13T02:22:27.416 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.410+0000 7f5425a4d6c0 1 -- 10.20.193.27:0/3547258024 wait complete. 2026-02-13T02:22:27.417 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.410+0000 7f1e04a096c0 1 -- 10.20.193.27:0/483797881 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1e00168d40 msgr2=0x7f1e001691e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.417 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.410+0000 7f1e04a096c0 1 --2- 10.20.193.27:0/483797881 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1e00168d40 0x7f1e001691e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.417 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.410+0000 7f1e04a096c0 1 -- 10.20.193.27:0/483797881 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1e00159a50 msgr2=0x7f1e00163980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.417 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.410+0000 7f1e04a096c0 1 --2- 10.20.193.27:0/483797881 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1e00159a50 0x7f1e00163980 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.417 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.410+0000 7f1e04a096c0 1 -- 10.20.193.27:0/483797881 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df00094c0 con 0x7f1e00062fd0 2026-02-13T02:22:27.417 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.410+0000 7f1df7fff6c0 1 --2- 10.20.193.27:0/483797881 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1e00159a50 0x7f1e00163980 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-13T02:22:27.417 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.411+0000 7f1e04a096c0 1 --2- 10.20.193.27:0/483797881 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1e00062fd0 0x7f1e00163440 secure :-1 s=READY pgs=53 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1df00098a0 tx=0x7f1df002fb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.417 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.411+0000 7f1df5ffb6c0 1 -- 10.20.193.27:0/483797881 <== mon.2 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1df0002ae0 con 0x7f1e00062fd0 2026-02-13T02:22:27.418 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.411+0000 7f5425a4d6c0 1 Processor -- start 2026-02-13T02:22:27.418 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.411+0000 7f5425a4d6c0 1 -- start start 2026-02-13T02:22:27.418 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.411+0000 7f5425a4d6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f542010f230 0x7f54200f4c20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.419 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.411+0000 7f5425a4d6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5420115440 0x7f5420175ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.419 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.411+0000 7f5425a4d6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f54200f5250 0x7f54200fa520 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.419 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.411+0000 7f1df5ffb6c0 1 -- 10.20.193.27:0/483797881 <== mon.2 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1df0005d50 con 0x7f1e00062fd0 2026-02-13T02:22:27.419 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.411+0000 7f1e06c986c0 1 -- 10.20.193.27:0/483797881 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e001578b0 con 0x7f1e00062fd0 2026-02-13T02:22:27.419 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.411+0000 7f1df5ffb6c0 1 -- 10.20.193.27:0/483797881 <== mon.2 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1df00312b0 con 0x7f1e00062fd0 2026-02-13T02:22:27.419 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.411+0000 7f1e06c986c0 1 -- 10.20.193.27:0/483797881 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e00157ce0 con 0x7f1e00062fd0 2026-02-13T02:22:27.420 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.411+0000 7f541effd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f542010f230 0x7f54200f4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.421 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.411+0000 7f541effd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f542010f230 0x7f54200f4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:53500/0 (socket says 10.20.193.27:53500) 2026-02-13T02:22:27.421 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.411+0000 7f541effd6c0 1 -- 10.20.193.27:0/807215352 learned_addr learned my addr 10.20.193.27:0/807215352 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:27.421 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.411+0000 7f5425a4d6c0 1 -- 10.20.193.27:0/807215352 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f54200f5940 con 0x7f542010f230 2026-02-13T02:22:27.421 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.411+0000 7f5425a4d6c0 1 -- 10.20.193.27:0/807215352 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f54200faa60 con 0x7f54200f5250 2026-02-13T02:22:27.421 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.411+0000 7f5425a4d6c0 1 -- 10.20.193.27:0/807215352 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f54200fac20 con 0x7f5420115440 2026-02-13T02:22:27.421 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.411+0000 7f541e7fc6c0 1 --2- 10.20.193.27:0/807215352 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5420115440 0x7f5420175ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.421 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.411+0000 7f541f7fe6c0 1 --2- 10.20.193.27:0/807215352 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f54200f5250 0x7f54200fa520 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.421 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.411+0000 7f541effd6c0 1 -- 10.20.193.27:0/807215352 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5420115440 msgr2=0x7f5420175ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.421 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.411+0000 7f541effd6c0 1 --2- 10.20.193.27:0/807215352 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5420115440 0x7f5420175ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.421 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.411+0000 7f541effd6c0 1 -- 10.20.193.27:0/807215352 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f54200f5250 msgr2=0x7f54200fa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.421 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.411+0000 7f541effd6c0 1 --2- 10.20.193.27:0/807215352 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f54200f5250 0x7f54200fa520 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.422 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.411+0000 7f541effd6c0 1 -- 10.20.193.27:0/807215352 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f541000a660 con 0x7f542010f230 2026-02-13T02:22:27.422 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.411+0000 7f541effd6c0 1 --2- 10.20.193.27:0/807215352 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f542010f230 0x7f54200f4c20 secure :-1 s=READY pgs=285 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f541400b710 tx=0x7f541400bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.422 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.411+0000 7f5424a476c0 1 -- 10.20.193.27:0/807215352 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5414004250 con 0x7f542010f230 2026-02-13T02:22:27.422 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.411+0000 7f5424a476c0 1 -- 10.20.193.27:0/807215352 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5414009070 con 0x7f542010f230 2026-02-13T02:22:27.422 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.411+0000 7f5424a476c0 1 -- 10.20.193.27:0/807215352 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f541400ccd0 con 0x7f542010f230 2026-02-13T02:22:27.422 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.412+0000 7f5425a4d6c0 1 -- 10.20.193.27:0/807215352 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54200faf00 con 0x7f542010f230 2026-02-13T02:22:27.422 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.411+0000 7f1dcf7fe6c0 1 -- 10.20.193.27:0/483797881 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e001596d0 con 0x7f1e00062fd0 2026-02-13T02:22:27.423 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.412+0000 7f5425a4d6c0 1 -- 10.20.193.27:0/807215352 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54200fb390 con 0x7f542010f230 2026-02-13T02:22:27.424 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.412+0000 7fabda6e06c0 1 -- 10.20.193.27:0/3978676771 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fabd41131c0 msgr2=0x7fabd4173b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.424 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.412+0000 7fabda6e06c0 1 --2- 10.20.193.27:0/3978676771 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fabd41131c0 0x7fabd4173b60 secure :-1 s=READY pgs=284 gs=1 cs=0 l=1 c_cookie=8308597734048723414 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fabc800aa70 tx=0x7fabc802fde0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.424 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.412+0000 7fabda6e06c0 1 -- 10.20.193.27:0/3978676771 shutdown_connections 2026-02-13T02:22:27.424 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.412+0000 7fabda6e06c0 1 --2- 10.20.193.27:0/3978676771 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fabd41131c0 0x7fabd4173b60 unknown :-1 s=CLOSED pgs=284 gs=1 cs=0 l=1 c_cookie=8308597734048723414 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.424 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.412+0000 7fabda6e06c0 1 --2- 10.20.193.27:0/3978676771 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fabd410d9a0 0x7fabd41129e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.424 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.412+0000 7fabda6e06c0 1 --2- 10.20.193.27:0/3978676771 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fabd410cfb0 0x7fabd410d380 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.424 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.412+0000 7fabda6e06c0 1 -- 10.20.193.27:0/3978676771 >> 10.20.193.27:0/3978676771 conn(0x7fabd40ea8b0 msgr2=0x7fabd410acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.424 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.412+0000 7fabda6e06c0 1 -- 10.20.193.27:0/3978676771 shutdown_connections 2026-02-13T02:22:27.424 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.412+0000 7fabda6e06c0 1 -- 10.20.193.27:0/3978676771 wait complete. 2026-02-13T02:22:27.424 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.412+0000 7fabda6e06c0 1 Processor -- start 2026-02-13T02:22:27.425 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.413+0000 7fabda6e06c0 1 -- start start 2026-02-13T02:22:27.425 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.413+0000 7fabda6e06c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fabd410cfb0 0x7fabd411af50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.425 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.413+0000 7fabda6e06c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fabd41131c0 0x7fabd411b490 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.425 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.413+0000 7fabda6e06c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fabd4120850 0x7fabd4120cf0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.425 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.413+0000 7fabda6e06c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fabd4121260 con 0x7fabd410cfb0 2026-02-13T02:22:27.425 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.413+0000 7fabda6e06c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fabd41213d0 con 0x7fabd4120850 2026-02-13T02:22:27.425 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.413+0000 7fabda6e06c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fabd4121590 con 0x7fabd41131c0 2026-02-13T02:22:27.425 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.413+0000 7fabd37fe6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fabd41131c0 0x7fabd411b490 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.425 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.413+0000 7fabd37fe6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fabd41131c0 0x7fabd411b490 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.107:3300/0 says I am v2:10.20.193.27:38360/0 (socket says 10.20.193.27:38360) 2026-02-13T02:22:27.425 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.413+0000 7fabd37fe6c0 1 -- 10.20.193.27:0/846292312 learned_addr learned my addr 10.20.193.27:0/846292312 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:27.426 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.413+0000 7f1df5ffb6c0 1 -- 10.20.193.27:0/483797881 <== mon.2 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f1df0002c40 con 0x7f1e00062fd0 2026-02-13T02:22:27.426 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.413+0000 7fabd3fff6c0 1 --2- 10.20.193.27:0/846292312 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fabd410cfb0 0x7fabd411af50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.426 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.413+0000 7f5425a4d6c0 1 -- 10.20.193.27:0/807215352 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5420110500 con 0x7f542010f230 2026-02-13T02:22:27.427 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.413+0000 7fabd3fff6c0 1 -- 10.20.193.27:0/846292312 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fabd41131c0 msgr2=0x7fabd411b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.427 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.413+0000 7fabd3fff6c0 1 --2- 10.20.193.27:0/846292312 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fabd41131c0 0x7fabd411b490 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.427 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.413+0000 7fabd3fff6c0 1 -- 10.20.193.27:0/846292312 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fabd4120850 msgr2=0x7fabd4120cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.427 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.413+0000 7fabd3fff6c0 1 --2- 10.20.193.27:0/846292312 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fabd4120850 0x7fabd4120cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.427 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.413+0000 7fabd3fff6c0 1 -- 10.20.193.27:0/846292312 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc800a660 con 0x7fabd410cfb0 2026-02-13T02:22:27.427 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.413+0000 7f1ef6b846c0 1 -- 10.20.193.27:0/3003429331 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1ef0115420 msgr2=0x7f1ef0175dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.428 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.413+0000 7f1ef6b846c0 1 --2- 10.20.193.27:0/3003429331 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1ef0115420 0x7f1ef0175dc0 secure :-1 s=READY pgs=286 gs=2 cs=0 l=1 c_cookie=2192516591707632124 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1ee40098d0 tx=0x7f1ee402ec60 comp rx=0 tx=0).stop 2026-02-13T02:22:27.428 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.413+0000 7fabd3fff6c0 1 --2- 10.20.193.27:0/846292312 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fabd410cfb0 0x7fabd411af50 secure :-1 s=READY pgs=287 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fabc000b6e0 tx=0x7fabc000bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.428 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.413+0000 7fabd17fa6c0 1 -- 10.20.193.27:0/846292312 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fabc0004a80 con 0x7fabd410cfb0 2026-02-13T02:22:27.429 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.414+0000 7f1df5ffb6c0 1 --2- 10.20.193.27:0/483797881 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f1ddc0b5690 0x7f1ddc0d5b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.429 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.414+0000 7f1df5ffb6c0 1 -- 10.20.193.27:0/483797881 <== mon.2 v2:10.20.193.28:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f1df0107b50 con 0x7f1e00062fd0 2026-02-13T02:22:27.429 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.414+0000 7f1df5ffb6c0 1 -- 10.20.193.27:0/483797881 <== mon.2 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f1df00ca570 con 0x7f1e00062fd0 2026-02-13T02:22:27.429 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.414+0000 7f1ef6b846c0 1 -- 10.20.193.27:0/3003429331 shutdown_connections 2026-02-13T02:22:27.429 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.414+0000 7f1ef6b846c0 1 --2- 10.20.193.27:0/3003429331 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1ef0115420 0x7f1ef0175dc0 unknown :-1 s=CLOSED pgs=286 gs=2 cs=0 l=1 c_cookie=2192516591707632124 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.430 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.414+0000 7f1ef6b846c0 1 --2- 10.20.193.27:0/3003429331 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1ef010fc00 0x7f1ef0114c40 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.430 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.414+0000 7f1ef6b846c0 1 --2- 10.20.193.27:0/3003429331 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1ef010f210 0x7f1ef010f5e0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.430 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.414+0000 7f1ef6b846c0 1 -- 10.20.193.27:0/3003429331 >> 10.20.193.27:0/3003429331 conn(0x7f1ef00ecb30 msgr2=0x7f1ef010cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.430 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.414+0000 7f1ef6b846c0 1 -- 10.20.193.27:0/3003429331 shutdown_connections 2026-02-13T02:22:27.430 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.414+0000 7fabd17fa6c0 1 -- 10.20.193.27:0/846292312 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fabc0009030 con 0x7fabd410cfb0 2026-02-13T02:22:27.430 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.414+0000 7fabda6e06c0 1 -- 10.20.193.27:0/846292312 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabd4161a40 con 0x7fabd410cfb0 2026-02-13T02:22:27.430 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.414+0000 7fabd17fa6c0 1 -- 10.20.193.27:0/846292312 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fabc000cc50 con 0x7fabd410cfb0 2026-02-13T02:22:27.431 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.414+0000 7f1df7fff6c0 1 --2- 10.20.193.27:0/483797881 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f1ddc0b5690 0x7f1ddc0d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.431 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.415+0000 7fabda6e06c0 1 -- 10.20.193.27:0/846292312 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabd4161f20 con 0x7fabd410cfb0 2026-02-13T02:22:27.431 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.415+0000 7f1ef6b846c0 1 -- 10.20.193.27:0/3003429331 wait complete. 2026-02-13T02:22:27.431 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.415+0000 7f1df7fff6c0 1 --2- 10.20.193.27:0/483797881 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f1ddc0b5690 0x7f1ddc0d5b40 secure :-1 s=READY pgs=200 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1e00169dd0 tx=0x7f1de8009310 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.432 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.414+0000 7f5424a476c0 1 -- 10.20.193.27:0/807215352 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f54140028c0 con 0x7f542010f230 2026-02-13T02:22:27.432 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.415+0000 7f5424a476c0 1 --2- 10.20.193.27:0/807215352 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f53f40b5710 0x7f53f40d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.432 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.415+0000 7f5424a476c0 1 -- 10.20.193.27:0/807215352 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f54140aa880 con 0x7f542010f230 2026-02-13T02:22:27.432 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.415+0000 7faba6ffd6c0 1 -- 10.20.193.27:0/846292312 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab9c005310 con 0x7fabd410cfb0 2026-02-13T02:22:27.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.415+0000 7f1ef6b846c0 1 Processor -- start 2026-02-13T02:22:27.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.416+0000 7f1ef6b846c0 1 -- start start 2026-02-13T02:22:27.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.416+0000 7f1ef6b846c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1ef010f210 0x7f1ef00f4be0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.434 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.416+0000 7f1ef6b846c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1ef0115420 0x7f1ef0175cc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.434 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.416+0000 7f1ef6b846c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1ef00f5210 0x7f1ef00fa490 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.434 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.416+0000 7f1ef6b846c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef00f5900 con 0x7f1ef00f5210 2026-02-13T02:22:27.434 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.416+0000 7f1ef6b846c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef00fa9d0 con 0x7f1ef0115420 2026-02-13T02:22:27.434 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.416+0000 7fabd17fa6c0 1 -- 10.20.193.27:0/846292312 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7fabc0004f20 con 0x7fabd410cfb0 2026-02-13T02:22:27.435 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.416+0000 7f1ef6b846c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef00fab90 con 0x7f1ef010f210 2026-02-13T02:22:27.435 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.416+0000 7f1eeffff6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1ef0115420 0x7f1ef0175cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.435 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.416+0000 7f1ef48f96c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1ef010f210 0x7f1ef00f4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.435 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.416+0000 7f1ef50fa6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1ef00f5210 0x7f1ef00fa490 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.435 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.416+0000 7f1ef48f96c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1ef010f210 0x7f1ef00f4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.107:3300/0 says I am v2:10.20.193.27:38362/0 (socket says 10.20.193.27:38362) 2026-02-13T02:22:27.435 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.416+0000 7f1eeffff6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1ef0115420 0x7f1ef0175cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.27:49264/0 (socket says 10.20.193.27:49264) 2026-02-13T02:22:27.435 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.416+0000 7f1ef48f96c0 1 -- 10.20.193.27:0/2669206531 learned_addr learned my addr 10.20.193.27:0/2669206531 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:27.435 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.416+0000 7f1ef50fa6c0 1 -- 10.20.193.27:0/2669206531 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1ef010f210 msgr2=0x7f1ef00f4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.435 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.416+0000 7f1ef50fa6c0 1 --2- 10.20.193.27:0/2669206531 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1ef010f210 0x7f1ef00f4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.435 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.416+0000 7f1ef50fa6c0 1 -- 10.20.193.27:0/2669206531 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1ef0115420 msgr2=0x7f1ef0175cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.435 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.416+0000 7f1ef50fa6c0 1 --2- 10.20.193.27:0/2669206531 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1ef0115420 0x7f1ef0175cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.435 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.416+0000 7f1ef50fa6c0 1 -- 10.20.193.27:0/2669206531 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee40094c0 con 0x7f1ef00f5210 2026-02-13T02:22:27.436 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.417+0000 7f1ef50fa6c0 1 --2- 10.20.193.27:0/2669206531 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1ef00f5210 0x7f1ef00fa490 secure :-1 s=READY pgs=288 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1ee40097f0 tx=0x7f1ee402fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.436 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.417+0000 7fabd17fa6c0 1 --2- 10.20.193.27:0/846292312 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7faba80b5710 0x7faba80d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.436 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.417+0000 7fabd37fe6c0 1 --2- 10.20.193.27:0/846292312 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7faba80b5710 0x7faba80d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.436 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.417+0000 7f541e7fc6c0 1 --2- 10.20.193.27:0/807215352 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f53f40b5710 0x7f53f40d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.436 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.417+0000 7f541e7fc6c0 1 --2- 10.20.193.27:0/807215352 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f53f40b5710 0x7f53f40d5bc0 secure :-1 s=READY pgs=201 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f54200fb5b0 tx=0x7f5408005f10 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.437 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.417+0000 7fabd37fe6c0 1 --2- 10.20.193.27:0/846292312 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7faba80b5710 0x7faba80d5bc0 secure :-1 s=READY pgs=202 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fabd41218e0 tx=0x7fabc4009260 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.437 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.418+0000 7f1eedffb6c0 1 -- 10.20.193.27:0/2669206531 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1ee4002ae0 con 0x7f1ef00f5210 2026-02-13T02:22:27.437 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.418+0000 7fabd17fa6c0 1 -- 10.20.193.27:0/846292312 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7fabc00aa7a0 con 0x7fabd410cfb0 2026-02-13T02:22:27.438 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.418+0000 7f1ef6b846c0 1 -- 10.20.193.27:0/2669206531 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ef00fae10 con 0x7f1ef00f5210 2026-02-13T02:22:27.438 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.418+0000 7f1ef6b846c0 1 -- 10.20.193.27:0/2669206531 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ef00fb310 con 0x7f1ef00f5210 2026-02-13T02:22:27.438 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.419+0000 7f1ef6b846c0 1 -- 10.20.193.27:0/2669206531 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ef01104e0 con 0x7f1ef00f5210 2026-02-13T02:22:27.438 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.418+0000 7f5424a476c0 1 -- 10.20.193.27:0/807215352 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f54140a6670 con 0x7f542010f230 2026-02-13T02:22:27.438 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.420+0000 7f1eedffb6c0 1 -- 10.20.193.27:0/2669206531 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1ee402fc30 con 0x7f1ef00f5210 2026-02-13T02:22:27.438 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.420+0000 7f1eedffb6c0 1 -- 10.20.193.27:0/2669206531 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1ee4007ea0 con 0x7f1ef00f5210 2026-02-13T02:22:27.438 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.422+0000 7fabd17fa6c0 1 -- 10.20.193.27:0/846292312 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fabc00a63f0 con 0x7fabd410cfb0 2026-02-13T02:22:27.439 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.422+0000 7f1eedffb6c0 1 -- 10.20.193.27:0/2669206531 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f1ee40071c0 con 0x7f1ef00f5210 2026-02-13T02:22:27.439 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.422+0000 7f1eedffb6c0 1 --2- 10.20.193.27:0/2669206531 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f1ecc0b5710 0x7f1ecc0d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.439 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.423+0000 7f1eedffb6c0 1 -- 10.20.193.27:0/2669206531 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f1ee4107aa0 con 0x7f1ef00f5210 2026-02-13T02:22:27.439 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.423+0000 7f1eedffb6c0 1 -- 10.20.193.27:0/2669206531 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f1ee41405e0 con 0x7f1ef00f5210 2026-02-13T02:22:27.439 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:27.440 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.424+0000 7f1ef48f96c0 1 --2- 10.20.193.27:0/2669206531 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f1ecc0b5710 0x7f1ecc0d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.440 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.424+0000 7f1ef48f96c0 1 --2- 10.20.193.27:0/2669206531 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f1ecc0b5710 0x7f1ecc0d5bc0 secure :-1 s=READY pgs=203 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1ee0006bc0 tx=0x7f1ee000a590 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.440 INFO:teuthology.orchestra.run.trial027.stdout:249108103177 2026-02-13T02:22:27.441 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.10 2026-02-13T02:22:27.484 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.482+0000 7fb0029f46c0 1 -- 10.20.193.27:0/779774875 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7faffc10e280 con 0x7faffc0f30d0 2026-02-13T02:22:27.484 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.482+0000 7faff97fa6c0 1 -- 10.20.193.27:0/779774875 <== mon.1 v2:10.20.193.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7faff00aa670 con 0x7faffc0f30d0 2026-02-13T02:22:27.484 INFO:teuthology.orchestra.run.trial027.stdout:111669149713 2026-02-13T02:22:27.485 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.483+0000 7fefc86eb6c0 1 -- 10.20.193.27:0/2538192014 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fefc010fbf0 con 0x7fefc0115410 2026-02-13T02:22:27.485 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.483+0000 7fefa77fe6c0 1 -- 10.20.193.27:0/2538192014 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fefb40ae780 con 0x7fefc0115410 2026-02-13T02:22:27.485 INFO:teuthology.orchestra.run.trial027.stdout:55834574868 2026-02-13T02:22:27.486 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.484+0000 7f3f723516c0 1 -- 10.20.193.27:0/787528539 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f3f6c1625f0 con 0x7f3f6c1556c0 2026-02-13T02:22:27.486 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.484+0000 7f3f697fa6c0 1 -- 10.20.193.27:0/787528539 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3f60003af0 con 0x7f3f6c1556c0 2026-02-13T02:22:27.486 INFO:teuthology.orchestra.run.trial027.stdout:30064771095 2026-02-13T02:22:27.487 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.485+0000 7fb0029f46c0 1 -- 10.20.193.27:0/779774875 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fafd00b57e0 msgr2=0x7fafd00d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.487 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.485+0000 7fb0029f46c0 1 --2- 10.20.193.27:0/779774875 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fafd00b57e0 0x7fafd00d5c90 secure :-1 s=READY pgs=197 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fafe8005980 tx=0x7fafe8005910 comp rx=0 tx=0).stop 2026-02-13T02:22:27.487 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.485+0000 7fb0029f46c0 1 -- 10.20.193.27:0/779774875 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7faffc0f30d0 msgr2=0x7faffc0f8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.487 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.485+0000 7fb0029f46c0 1 --2- 10.20.193.27:0/779774875 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7faffc0f30d0 0x7faffc0f8350 secure :-1 s=READY pgs=55 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7faff000c800 tx=0x7faff000ccd0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.487 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.485+0000 7f8332f016c0 1 -- 10.20.193.27:0/564497662 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f832c10fc20 msgr2=0x7f832c114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.487 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.485+0000 7f8332f016c0 1 --2- 10.20.193.27:0/564497662 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f832c10fc20 0x7f832c114c60 secure :-1 s=READY pgs=289 gs=1 cs=0 l=1 c_cookie=16087224800701421560 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f83140099a0 tx=0x7f831402ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:27.487 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.485+0000 7f8332f016c0 1 -- 10.20.193.27:0/564497662 shutdown_connections 2026-02-13T02:22:27.488 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.485+0000 7f8332f016c0 1 --2- 10.20.193.27:0/564497662 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f832c115440 0x7f832c175de0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.488 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.485+0000 7f8332f016c0 1 --2- 10.20.193.27:0/564497662 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f832c10fc20 0x7f832c114c60 unknown :-1 s=CLOSED pgs=289 gs=1 cs=0 l=1 c_cookie=16087224800701421560 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.488 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.485+0000 7f8332f016c0 1 --2- 10.20.193.27:0/564497662 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f832c10f230 0x7f832c10f600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.488 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.485+0000 7f8332f016c0 1 -- 10.20.193.27:0/564497662 >> 10.20.193.27:0/564497662 conn(0x7f832c0ecb30 msgr2=0x7f832c10cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.488 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.485+0000 7fb0029f46c0 1 -- 10.20.193.27:0/779774875 shutdown_connections 2026-02-13T02:22:27.488 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.485+0000 7fb0029f46c0 1 --2- 10.20.193.27:0/779774875 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fafd00b57e0 0x7fafd00d5c90 unknown :-1 s=CLOSED pgs=197 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.488 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.485+0000 7fb0029f46c0 1 --2- 10.20.193.27:0/779774875 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7faffc0f30d0 0x7faffc0f8350 unknown :-1 s=CLOSED pgs=55 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.488 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.485+0000 7fb0029f46c0 1 --2- 10.20.193.27:0/779774875 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7faffc1131c0 0x7faffc173a60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.489 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.485+0000 7fb0029f46c0 1 --2- 10.20.193.27:0/779774875 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faffc10cfb0 0x7faffc0f2aa0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.489 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.485+0000 7fb0029f46c0 1 -- 10.20.193.27:0/779774875 >> 10.20.193.27:0/779774875 conn(0x7faffc0ea8b0 msgr2=0x7faffc10b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.489 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.485+0000 7f8332f016c0 1 -- 10.20.193.27:0/564497662 shutdown_connections 2026-02-13T02:22:27.489 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.485+0000 7f8332f016c0 1 -- 10.20.193.27:0/564497662 wait complete. 2026-02-13T02:22:27.489 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.485+0000 7fb0029f46c0 1 -- 10.20.193.27:0/779774875 shutdown_connections 2026-02-13T02:22:27.489 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.485+0000 7fb0029f46c0 1 -- 10.20.193.27:0/779774875 wait complete. 2026-02-13T02:22:27.490 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.486+0000 7f8332f016c0 1 Processor -- start 2026-02-13T02:22:27.490 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.486+0000 7f8332f016c0 1 -- start start 2026-02-13T02:22:27.490 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.486+0000 7f8332f016c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f832c10f230 0x7f832c11d2f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.490 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.486+0000 7f8332f016c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f832c115440 0x7f832c11d830 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.491 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.486+0000 7f8332f016c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f832c122bf0 0x7f832c123090 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.491 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.486+0000 7f8332f016c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f832c123600 con 0x7f832c10f230 2026-02-13T02:22:27.491 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.486+0000 7f8332f016c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f832c123770 con 0x7f832c122bf0 2026-02-13T02:22:27.491 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.486+0000 7f8332f016c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f832c123930 con 0x7f832c115440 2026-02-13T02:22:27.491 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.486+0000 7f8330c726c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f832c10f230 0x7f832c11d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.491 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.486+0000 7f8330c726c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f832c10f230 0x7f832c11d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:53542/0 (socket says 10.20.193.27:53542) 2026-02-13T02:22:27.491 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.486+0000 7f8330c726c0 1 -- 10.20.193.27:0/2663354391 learned_addr learned my addr 10.20.193.27:0/2663354391 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:27.491 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.486+0000 7f83314736c0 1 --2- 10.20.193.27:0/2663354391 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f832c122bf0 0x7f832c123090 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.491 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.486+0000 7f8323fff6c0 1 --2- 10.20.193.27:0/2663354391 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f832c115440 0x7f832c11d830 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.492 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.486+0000 7f3f723516c0 1 -- 10.20.193.27:0/787528539 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3f400b5690 msgr2=0x7f3f400d5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.492 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.486+0000 7f3f723516c0 1 --2- 10.20.193.27:0/787528539 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3f400b5690 0x7f3f400d5b40 secure :-1 s=READY pgs=199 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3f58004750 tx=0x7f3f58006d50 comp rx=0 tx=0).stop 2026-02-13T02:22:27.492 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.487+0000 7f3f723516c0 1 -- 10.20.193.27:0/787528539 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3f6c1556c0 msgr2=0x7f3f6c166da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.492 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.487+0000 7f3f723516c0 1 --2- 10.20.193.27:0/787528539 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3f6c1556c0 0x7f3f6c166da0 secure :-1 s=READY pgs=282 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3f6000c6c0 tx=0x7f3f6000cb90 comp rx=0 tx=0).stop 2026-02-13T02:22:27.492 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.487+0000 7f3f723516c0 1 -- 10.20.193.27:0/787528539 shutdown_connections 2026-02-13T02:22:27.492 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.487+0000 7f3f723516c0 1 --2- 10.20.193.27:0/787528539 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3f400b5690 0x7f3f400d5b40 unknown :-1 s=CLOSED pgs=199 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.492 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.487+0000 7f3f723516c0 1 --2- 10.20.193.27:0/787528539 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3f6c1556c0 0x7f3f6c166da0 unknown :-1 s=CLOSED pgs=282 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.492 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.487+0000 7f3f723516c0 1 --2- 10.20.193.27:0/787528539 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3f6c061050 0x7f3f6c1619e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.492 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.487+0000 7f3f723516c0 1 --2- 10.20.193.27:0/787528539 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3f6c060740 0x7f3f6c1614a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.492 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.487+0000 7f3f723516c0 1 -- 10.20.193.27:0/787528539 >> 10.20.193.27:0/787528539 conn(0x7f3f6c0ea8b0 msgr2=0x7f3f6c10ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.493 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.487+0000 7f3f723516c0 1 -- 10.20.193.27:0/787528539 shutdown_connections 2026-02-13T02:22:27.493 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.487+0000 7f3f723516c0 1 -- 10.20.193.27:0/787528539 wait complete. 2026-02-13T02:22:27.493 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.487+0000 7fefc86eb6c0 1 -- 10.20.193.27:0/2538192014 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fef9c0b56f0 msgr2=0x7fef9c0d5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.493 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.487+0000 7fefc86eb6c0 1 --2- 10.20.193.27:0/2538192014 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fef9c0b56f0 0x7fef9c0d5ba0 secure :-1 s=READY pgs=198 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fefb00098c0 tx=0x7fefb0005c20 comp rx=0 tx=0).stop 2026-02-13T02:22:27.493 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.487+0000 7fefc86eb6c0 1 -- 10.20.193.27:0/2538192014 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fefc0115410 msgr2=0x7fefc011d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.493 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.487+0000 7fefc86eb6c0 1 --2- 10.20.193.27:0/2538192014 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fefc0115410 0x7fefc011d770 secure :-1 s=READY pgs=281 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fefb4009e00 tx=0x7fefb400ba90 comp rx=0 tx=0).stop 2026-02-13T02:22:27.493 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.487+0000 7fefc86eb6c0 1 -- 10.20.193.27:0/2538192014 shutdown_connections 2026-02-13T02:22:27.493 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.487+0000 7fefc86eb6c0 1 --2- 10.20.193.27:0/2538192014 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fef9c0b56f0 0x7fef9c0d5ba0 unknown :-1 s=CLOSED pgs=198 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.493 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.487+0000 7fefc86eb6c0 1 --2- 10.20.193.27:0/2538192014 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fefc0122b30 0x7fefc0122fd0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.493 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.487+0000 7fefc86eb6c0 1 --2- 10.20.193.27:0/2538192014 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fefc0115410 0x7fefc011d770 unknown :-1 s=CLOSED pgs=281 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.494 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.487+0000 7fefc86eb6c0 1 --2- 10.20.193.27:0/2538192014 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fefc010f200 0x7fefc011d230 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.494 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.487+0000 7fefc86eb6c0 1 -- 10.20.193.27:0/2538192014 >> 10.20.193.27:0/2538192014 conn(0x7fefc00ecb00 msgr2=0x7fefc010cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.494 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.487+0000 7fefc86eb6c0 1 -- 10.20.193.27:0/2538192014 shutdown_connections 2026-02-13T02:22:27.494 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.487+0000 7fefc86eb6c0 1 -- 10.20.193.27:0/2538192014 wait complete. 2026-02-13T02:22:27.495 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.486+0000 7f8330c726c0 1 -- 10.20.193.27:0/2663354391 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f832c115440 msgr2=0x7f832c11d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.495 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.486+0000 7f8330c726c0 1 --2- 10.20.193.27:0/2663354391 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f832c115440 0x7f832c11d830 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.495 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.486+0000 7f8330c726c0 1 -- 10.20.193.27:0/2663354391 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f832c122bf0 msgr2=0x7f832c123090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.495 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.486+0000 7f8330c726c0 1 --2- 10.20.193.27:0/2663354391 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f832c122bf0 0x7f832c123090 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.495 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.486+0000 7f8330c726c0 1 -- 10.20.193.27:0/2663354391 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831c0094c0 con 0x7f832c10f230 2026-02-13T02:22:27.495 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.486+0000 7f8330c726c0 1 --2- 10.20.193.27:0/2663354391 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f832c10f230 0x7f832c11d2f0 secure :-1 s=READY pgs=290 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f831c00ba30 tx=0x7f831c00bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.495 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.486+0000 7f8321ffb6c0 1 -- 10.20.193.27:0/2663354391 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f831c0044a0 con 0x7f832c10f230 2026-02-13T02:22:27.495 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.486+0000 7f8321ffb6c0 1 -- 10.20.193.27:0/2663354391 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f831c009030 con 0x7f832c10f230 2026-02-13T02:22:27.496 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.486+0000 7f8332f016c0 1 -- 10.20.193.27:0/2663354391 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8314009590 con 0x7f832c10f230 2026-02-13T02:22:27.496 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.486+0000 7f8321ffb6c0 1 -- 10.20.193.27:0/2663354391 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f831c0071c0 con 0x7f832c10f230 2026-02-13T02:22:27.496 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.487+0000 7f8332f016c0 1 -- 10.20.193.27:0/2663354391 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f832c1640e0 con 0x7f832c10f230 2026-02-13T02:22:27.496 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.487+0000 7f8332f016c0 1 -- 10.20.193.27:0/2663354391 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f832c110500 con 0x7f832c10f230 2026-02-13T02:22:27.496 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.489+0000 7f8321ffb6c0 1 -- 10.20.193.27:0/2663354391 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f831c0026b0 con 0x7f832c10f230 2026-02-13T02:22:27.496 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.489+0000 7f8321ffb6c0 1 --2- 10.20.193.27:0/2663354391 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f83080b5690 0x7f83080d5b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.496 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.490+0000 7f8323fff6c0 1 --2- 10.20.193.27:0/2663354391 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f83080b5690 0x7f83080d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.496 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.490+0000 7f8321ffb6c0 1 -- 10.20.193.27:0/2663354391 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f831c0e3cd0 con 0x7f832c10f230 2026-02-13T02:22:27.496 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.490+0000 7f8323fff6c0 1 --2- 10.20.193.27:0/2663354391 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f83080b5690 0x7f83080d5b40 secure :-1 s=READY pgs=204 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f83140092e0 tx=0x7f8314005b00 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.496 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.490+0000 7f8321ffb6c0 1 -- 10.20.193.27:0/2663354391 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f831c0a6730 con 0x7f832c10f230 2026-02-13T02:22:27.496 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:27 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3678262786' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-13T02:22:27.496 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:27 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3132368379' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-13T02:22:27.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:27 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3678262786' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-13T02:22:27.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:27 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3132368379' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-13T02:22:27.503 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.499+0000 7f40614d16c0 1 -- 10.20.193.27:0/2301416184 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f405c10fc20 msgr2=0x7f405c114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.503 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.499+0000 7f40614d16c0 1 --2- 10.20.193.27:0/2301416184 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f405c10fc20 0x7f405c114c60 secure :-1 s=READY pgs=291 gs=1 cs=0 l=1 c_cookie=15212713459900891668 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f40440099a0 tx=0x7f404402ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:27.503 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.499+0000 7f40614d16c0 1 -- 10.20.193.27:0/2301416184 shutdown_connections 2026-02-13T02:22:27.504 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.499+0000 7f40614d16c0 1 --2- 10.20.193.27:0/2301416184 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f405c115440 0x7f405c175de0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.504 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.499+0000 7f40614d16c0 1 --2- 10.20.193.27:0/2301416184 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f405c10fc20 0x7f405c114c60 unknown :-1 s=CLOSED pgs=291 gs=1 cs=0 l=1 c_cookie=15212713459900891668 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.504 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.499+0000 7f40614d16c0 1 --2- 10.20.193.27:0/2301416184 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f405c10f230 0x7f405c10f600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.504 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.499+0000 7f40614d16c0 1 -- 10.20.193.27:0/2301416184 >> 10.20.193.27:0/2301416184 conn(0x7f405c0ecb30 msgr2=0x7f405c10cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.504 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.499+0000 7f40614d16c0 1 -- 10.20.193.27:0/2301416184 shutdown_connections 2026-02-13T02:22:27.504 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.499+0000 7f40614d16c0 1 -- 10.20.193.27:0/2301416184 wait complete. 2026-02-13T02:22:27.504 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.500+0000 7f40614d16c0 1 Processor -- start 2026-02-13T02:22:27.504 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.500+0000 7f40614d16c0 1 -- start start 2026-02-13T02:22:27.504 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.500+0000 7f40614d16c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f405c10f230 0x7f405c0f4c40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.504 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.500+0000 7f40614d16c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f405c115440 0x7f405c175ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.505 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.500+0000 7f40614d16c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f405c0f5270 0x7f405c0fa540 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.505 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.500+0000 7f40614d16c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f405c0f5960 con 0x7f405c0f5270 2026-02-13T02:22:27.505 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.500+0000 7f40614d16c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f405c0faa80 con 0x7f405c10f230 2026-02-13T02:22:27.505 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.500+0000 7f40614d16c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f405c0fac40 con 0x7f405c115440 2026-02-13T02:22:27.505 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.500+0000 7f405b7fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f405c0f5270 0x7f405c0fa540 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.505 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.500+0000 7f405affd6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f405c10f230 0x7f405c0f4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.505 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.500+0000 7f405b7fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f405c0f5270 0x7f405c0fa540 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:53560/0 (socket says 10.20.193.27:53560) 2026-02-13T02:22:27.505 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.500+0000 7f405affd6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f405c10f230 0x7f405c0f4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.27:49300/0 (socket says 10.20.193.27:49300) 2026-02-13T02:22:27.505 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.500+0000 7f405b7fe6c0 1 -- 10.20.193.27:0/1465769140 learned_addr learned my addr 10.20.193.27:0/1465769140 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:27.505 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.500+0000 7f405a7fc6c0 1 --2- 10.20.193.27:0/1465769140 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f405c115440 0x7f405c175ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.505 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.500+0000 7f405b7fe6c0 1 -- 10.20.193.27:0/1465769140 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f405c115440 msgr2=0x7f405c175ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.506 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.500+0000 7f405b7fe6c0 1 --2- 10.20.193.27:0/1465769140 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f405c115440 0x7f405c175ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.506 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.500+0000 7f405b7fe6c0 1 -- 10.20.193.27:0/1465769140 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f405c10f230 msgr2=0x7f405c0f4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.506 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.500+0000 7f405b7fe6c0 1 --2- 10.20.193.27:0/1465769140 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f405c10f230 0x7f405c0f4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.506 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.501+0000 7f405b7fe6c0 1 -- 10.20.193.27:0/1465769140 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40500094c0 con 0x7f405c0f5270 2026-02-13T02:22:27.506 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.501+0000 7f405b7fe6c0 1 --2- 10.20.193.27:0/1465769140 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f405c0f5270 0x7f405c0fa540 secure :-1 s=READY pgs=292 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f404c00a820 tx=0x7f404c00ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.506 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.501+0000 7f403bfff6c0 1 -- 10.20.193.27:0/1465769140 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f404c003c40 con 0x7f405c0f5270 2026-02-13T02:22:27.506 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.501+0000 7f40614d16c0 1 -- 10.20.193.27:0/1465769140 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f404c00a5b0 con 0x7f405c0f5270 2026-02-13T02:22:27.506 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.501+0000 7f40614d16c0 1 -- 10.20.193.27:0/1465769140 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4044009590 con 0x7f405c0f5270 2026-02-13T02:22:27.506 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.501+0000 7f403bfff6c0 1 -- 10.20.193.27:0/1465769140 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f404c00a070 con 0x7f405c0f5270 2026-02-13T02:22:27.506 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.501+0000 7f403bfff6c0 1 -- 10.20.193.27:0/1465769140 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f404c0081c0 con 0x7f405c0f5270 2026-02-13T02:22:27.506 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.501+0000 7f40614d16c0 1 -- 10.20.193.27:0/1465769140 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f405c110500 con 0x7f405c0f5270 2026-02-13T02:22:27.507 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.504+0000 7f403bfff6c0 1 -- 10.20.193.27:0/1465769140 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f404c005860 con 0x7f405c0f5270 2026-02-13T02:22:27.508 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.505+0000 7f403bfff6c0 1 --2- 10.20.193.27:0/1465769140 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f40300b5690 0x7f40300d5b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.508 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.505+0000 7f403bfff6c0 1 -- 10.20.193.27:0/1465769140 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f404c0e4b40 con 0x7f405c0f5270 2026-02-13T02:22:27.508 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.505+0000 7f405affd6c0 1 --2- 10.20.193.27:0/1465769140 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f40300b5690 0x7f40300d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.508 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.505+0000 7f403bfff6c0 1 -- 10.20.193.27:0/1465769140 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f404c11d5e0 con 0x7f405c0f5270 2026-02-13T02:22:27.508 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.506+0000 7f405affd6c0 1 --2- 10.20.193.27:0/1465769140 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f40300b5690 0x7f40300d5b40 secure :-1 s=READY pgs=205 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4050004750 tx=0x7f4050009260 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.529 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.527+0000 7f1dcf7fe6c0 1 -- 10.20.193.27:0/483797881 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f1e00063ec0 con 0x7f1e00062fd0 2026-02-13T02:22:27.529 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.528+0000 7f1df5ffb6c0 1 -- 10.20.193.27:0/483797881 <== mon.2 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f1df00cd720 con 0x7f1e00062fd0 2026-02-13T02:22:27.529 INFO:teuthology.orchestra.run.trial027.stdout:279172874246 2026-02-13T02:22:27.532 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.530+0000 7f1dcf7fe6c0 1 -- 10.20.193.27:0/483797881 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f1ddc0b5690 msgr2=0x7f1ddc0d5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.532 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.530+0000 7f1dcf7fe6c0 1 --2- 10.20.193.27:0/483797881 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f1ddc0b5690 0x7f1ddc0d5b40 secure :-1 s=READY pgs=200 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1e00169dd0 tx=0x7f1de8009310 comp rx=0 tx=0).stop 2026-02-13T02:22:27.532 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.530+0000 7f1dcf7fe6c0 1 -- 10.20.193.27:0/483797881 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1e00062fd0 msgr2=0x7f1e00163440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.532 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.530+0000 7f1dcf7fe6c0 1 --2- 10.20.193.27:0/483797881 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1e00062fd0 0x7f1e00163440 secure :-1 s=READY pgs=53 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1df00098a0 tx=0x7f1df002fb20 comp rx=0 tx=0).stop 2026-02-13T02:22:27.532 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.530+0000 7f1dcf7fe6c0 1 -- 10.20.193.27:0/483797881 shutdown_connections 2026-02-13T02:22:27.532 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.530+0000 7f1dcf7fe6c0 1 --2- 10.20.193.27:0/483797881 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f1ddc0b5690 0x7f1ddc0d5b40 unknown :-1 s=CLOSED pgs=200 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.532 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.530+0000 7f1dcf7fe6c0 1 --2- 10.20.193.27:0/483797881 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1e00168d40 0x7f1e001691e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.532 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.530+0000 7f1dcf7fe6c0 1 --2- 10.20.193.27:0/483797881 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1e00159a50 0x7f1e00163980 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.532 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.530+0000 7f1dcf7fe6c0 1 --2- 10.20.193.27:0/483797881 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1e00062fd0 0x7f1e00163440 unknown :-1 s=CLOSED pgs=53 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.532 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.530+0000 7f1dcf7fe6c0 1 -- 10.20.193.27:0/483797881 >> 10.20.193.27:0/483797881 conn(0x7f1e000ecb00 msgr2=0x7f1e0010cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.533 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.530+0000 7f1dcf7fe6c0 1 -- 10.20.193.27:0/483797881 shutdown_connections 2026-02-13T02:22:27.533 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.530+0000 7f1dcf7fe6c0 1 -- 10.20.193.27:0/483797881 wait complete. 2026-02-13T02:22:27.536 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.534+0000 7f5425a4d6c0 1 -- 10.20.193.27:0/807215352 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f54200fb390 con 0x7f542010f230 2026-02-13T02:22:27.536 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.535+0000 7f5424a476c0 1 -- 10.20.193.27:0/807215352 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f54200fb390 con 0x7f542010f230 2026-02-13T02:22:27.536 INFO:teuthology.orchestra.run.trial027.stdout:81604378643 2026-02-13T02:22:27.539 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.537+0000 7f5425a4d6c0 1 -- 10.20.193.27:0/807215352 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f53f40b5710 msgr2=0x7f53f40d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.539 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.537+0000 7f5425a4d6c0 1 --2- 10.20.193.27:0/807215352 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f53f40b5710 0x7f53f40d5bc0 secure :-1 s=READY pgs=201 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f54200fb5b0 tx=0x7f5408005f10 comp rx=0 tx=0).stop 2026-02-13T02:22:27.539 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.537+0000 7f5425a4d6c0 1 -- 10.20.193.27:0/807215352 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f542010f230 msgr2=0x7f54200f4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.539 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.537+0000 7f5425a4d6c0 1 --2- 10.20.193.27:0/807215352 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f542010f230 0x7f54200f4c20 secure :-1 s=READY pgs=285 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f541400b710 tx=0x7f541400bbe0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.539 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.537+0000 7f5425a4d6c0 1 -- 10.20.193.27:0/807215352 shutdown_connections 2026-02-13T02:22:27.539 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.537+0000 7f5425a4d6c0 1 --2- 10.20.193.27:0/807215352 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f53f40b5710 0x7f53f40d5bc0 unknown :-1 s=CLOSED pgs=201 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.539 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.537+0000 7f5425a4d6c0 1 --2- 10.20.193.27:0/807215352 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f54200f5250 0x7f54200fa520 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.539 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.537+0000 7f5425a4d6c0 1 --2- 10.20.193.27:0/807215352 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5420115440 0x7f5420175ce0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.539 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.537+0000 7f5425a4d6c0 1 --2- 10.20.193.27:0/807215352 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f542010f230 0x7f54200f4c20 unknown :-1 s=CLOSED pgs=285 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.539 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.537+0000 7f5425a4d6c0 1 -- 10.20.193.27:0/807215352 >> 10.20.193.27:0/807215352 conn(0x7f54200ecb30 msgr2=0x7f542010d830 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.539 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.537+0000 7f5425a4d6c0 1 -- 10.20.193.27:0/807215352 shutdown_connections 2026-02-13T02:22:27.540 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.537+0000 7f5425a4d6c0 1 -- 10.20.193.27:0/807215352 wait complete. 2026-02-13T02:22:27.542 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.541+0000 7faba6ffd6c0 1 -- 10.20.193.27:0/846292312 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fab9c005890 con 0x7fabd410cfb0 2026-02-13T02:22:27.543 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.541+0000 7fabd17fa6c0 1 -- 10.20.193.27:0/846292312 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fabc00a95a0 con 0x7fabd410cfb0 2026-02-13T02:22:27.543 INFO:teuthology.orchestra.run.trial027.stdout:171798691852 2026-02-13T02:22:27.544 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574869 got 55834574868 for osd.2 2026-02-13T02:22:27.545 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.542+0000 7f1ef6b846c0 1 -- 10.20.193.27:0/2669206531 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f1ef010fbb0 con 0x7f1ef00f5210 2026-02-13T02:22:27.546 INFO:tasks.cephadm.ceph_manager.ceph:need seq 30064771096 got 30064771095 for osd.0 2026-02-13T02:22:27.546 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.542+0000 7f1eedffb6c0 1 -- 10.20.193.27:0/2669206531 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1ee40ca4c0 con 0x7f1ef00f5210 2026-02-13T02:22:27.546 INFO:teuthology.orchestra.run.trial027.stdout:201863462923 2026-02-13T02:22:27.547 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.544+0000 7faba6ffd6c0 1 -- 10.20.193.27:0/846292312 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7faba80b5710 msgr2=0x7faba80d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.547 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.544+0000 7faba6ffd6c0 1 --2- 10.20.193.27:0/846292312 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7faba80b5710 0x7faba80d5bc0 secure :-1 s=READY pgs=202 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fabd41218e0 tx=0x7fabc4009260 comp rx=0 tx=0).stop 2026-02-13T02:22:27.547 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.544+0000 7faba6ffd6c0 1 -- 10.20.193.27:0/846292312 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fabd410cfb0 msgr2=0x7fabd411af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.547 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.544+0000 7faba6ffd6c0 1 --2- 10.20.193.27:0/846292312 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fabd410cfb0 0x7fabd411af50 secure :-1 s=READY pgs=287 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fabc000b6e0 tx=0x7fabc000bbb0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.547 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.544+0000 7faba6ffd6c0 1 -- 10.20.193.27:0/846292312 shutdown_connections 2026-02-13T02:22:27.548 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.544+0000 7faba6ffd6c0 1 --2- 10.20.193.27:0/846292312 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7faba80b5710 0x7faba80d5bc0 unknown :-1 s=CLOSED pgs=202 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.548 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.544+0000 7faba6ffd6c0 1 --2- 10.20.193.27:0/846292312 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fabd4120850 0x7fabd4120cf0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.548 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.544+0000 7faba6ffd6c0 1 --2- 10.20.193.27:0/846292312 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fabd41131c0 0x7fabd411b490 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.548 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.544+0000 7faba6ffd6c0 1 --2- 10.20.193.27:0/846292312 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fabd410cfb0 0x7fabd411af50 unknown :-1 s=CLOSED pgs=287 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.549 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.545+0000 7f1ef6b846c0 1 -- 10.20.193.27:0/2669206531 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f1ecc0b5710 msgr2=0x7f1ecc0d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.549 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.545+0000 7f1ef6b846c0 1 --2- 10.20.193.27:0/2669206531 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f1ecc0b5710 0x7f1ecc0d5bc0 secure :-1 s=READY pgs=203 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1ee0006bc0 tx=0x7f1ee000a590 comp rx=0 tx=0).stop 2026-02-13T02:22:27.549 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.545+0000 7f1ef6b846c0 1 -- 10.20.193.27:0/2669206531 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1ef00f5210 msgr2=0x7f1ef00fa490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.549 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.545+0000 7f1ef6b846c0 1 --2- 10.20.193.27:0/2669206531 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1ef00f5210 0x7f1ef00fa490 secure :-1 s=READY pgs=288 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1ee40097f0 tx=0x7f1ee402fac0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.549 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.545+0000 7f1ef6b846c0 1 -- 10.20.193.27:0/2669206531 shutdown_connections 2026-02-13T02:22:27.549 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.545+0000 7f1ef6b846c0 1 --2- 10.20.193.27:0/2669206531 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f1ecc0b5710 0x7f1ecc0d5bc0 unknown :-1 s=CLOSED pgs=203 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.549 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.545+0000 7f1ef6b846c0 1 --2- 10.20.193.27:0/2669206531 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1ef00f5210 0x7f1ef00fa490 unknown :-1 s=CLOSED pgs=288 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.549 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.545+0000 7f1ef6b846c0 1 --2- 10.20.193.27:0/2669206531 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1ef0115420 0x7f1ef0175cc0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.549 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.545+0000 7f1ef6b846c0 1 --2- 10.20.193.27:0/2669206531 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1ef010f210 0x7f1ef00f4be0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.550 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.545+0000 7f1ef6b846c0 1 -- 10.20.193.27:0/2669206531 >> 10.20.193.27:0/2669206531 conn(0x7f1ef00ecb30 msgr2=0x7f1ef010d830 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.550 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.545+0000 7f1ef6b846c0 1 -- 10.20.193.27:0/2669206531 shutdown_connections 2026-02-13T02:22:27.550 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.545+0000 7f1ef6b846c0 1 -- 10.20.193.27:0/2669206531 wait complete. 2026-02-13T02:22:27.550 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.544+0000 7faba6ffd6c0 1 -- 10.20.193.27:0/846292312 >> 10.20.193.27:0/846292312 conn(0x7fabd40ea8b0 msgr2=0x7fabd410ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.550 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.544+0000 7faba6ffd6c0 1 -- 10.20.193.27:0/846292312 shutdown_connections 2026-02-13T02:22:27.550 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.545+0000 7faba6ffd6c0 1 -- 10.20.193.27:0/846292312 wait complete. 2026-02-13T02:22:27.552 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.551+0000 7fea87a8c6c0 1 -- 10.20.193.27:0/870634712 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fea8010c4e0 msgr2=0x7fea8010c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.552 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.551+0000 7fea87a8c6c0 1 --2- 10.20.193.27:0/870634712 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fea8010c4e0 0x7fea8010c980 secure :-1 s=READY pgs=293 gs=1 cs=0 l=1 c_cookie=15728771292206245876 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fea7c00aa40 tx=0x7fea7c02fde0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.553 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.551+0000 7fea87a8c6c0 1 -- 10.20.193.27:0/870634712 shutdown_connections 2026-02-13T02:22:27.553 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.551+0000 7fea87a8c6c0 1 --2- 10.20.193.27:0/870634712 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fea8010c4e0 0x7fea8010c980 unknown :-1 s=CLOSED pgs=293 gs=1 cs=0 l=1 c_cookie=15728771292206245876 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.553 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.551+0000 7fea87a8c6c0 1 --2- 10.20.193.27:0/870634712 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fea80112fc0 0x7fea8010b730 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.553 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.551+0000 7fea87a8c6c0 1 --2- 10.20.193.27:0/870634712 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fea80112b10 0x7fea8010b1f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.553 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.551+0000 7fea87a8c6c0 1 -- 10.20.193.27:0/870634712 >> 10.20.193.27:0/870634712 conn(0x7fea80063af0 msgr2=0x7fea800642b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.553 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.551+0000 7fea87a8c6c0 1 -- 10.20.193.27:0/870634712 shutdown_connections 2026-02-13T02:22:27.553 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.551+0000 7fea87a8c6c0 1 -- 10.20.193.27:0/870634712 wait complete. 2026-02-13T02:22:27.555 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.552+0000 7fea87a8c6c0 1 Processor -- start 2026-02-13T02:22:27.555 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.552+0000 7fea87a8c6c0 1 -- start start 2026-02-13T02:22:27.555 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.552+0000 7fea87a8c6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fea8010c4e0 0x7fea80122c80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.555 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.552+0000 7fea87a8c6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fea80112b10 0x7fea801231c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.555 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.552+0000 7fea87a8c6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fea80112fc0 0x7fea80128580 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.555 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.552+0000 7fea87a8c6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fea80128bd0 con 0x7fea80112fc0 2026-02-13T02:22:27.556 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.552+0000 7fea87a8c6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fea80128d40 con 0x7fea80112b10 2026-02-13T02:22:27.556 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.552+0000 7fea87a8c6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fea80128f00 con 0x7fea8010c4e0 2026-02-13T02:22:27.556 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.552+0000 7fea85ffe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fea80112fc0 0x7fea80128580 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.556 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.552+0000 7fea85ffe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fea80112fc0 0x7fea80128580 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:53594/0 (socket says 10.20.193.27:53594) 2026-02-13T02:22:27.556 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.552+0000 7fea85ffe6c0 1 -- 10.20.193.27:0/3358083149 learned_addr learned my addr 10.20.193.27:0/3358083149 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:27.556 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.552+0000 7fea857fd6c0 1 --2- 10.20.193.27:0/3358083149 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fea8010c4e0 0x7fea80122c80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.556 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.552+0000 7fea84ffc6c0 1 --2- 10.20.193.27:0/3358083149 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fea80112b10 0x7fea801231c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.556 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.552+0000 7fea85ffe6c0 1 -- 10.20.193.27:0/3358083149 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fea8010c4e0 msgr2=0x7fea80122c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.556 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.552+0000 7fea85ffe6c0 1 --2- 10.20.193.27:0/3358083149 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fea8010c4e0 0x7fea80122c80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.556 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.552+0000 7fea85ffe6c0 1 -- 10.20.193.27:0/3358083149 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fea80112b10 msgr2=0x7fea801231c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.556 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.552+0000 7fea85ffe6c0 1 --2- 10.20.193.27:0/3358083149 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fea80112b10 0x7fea801231c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.556 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.552+0000 7fea85ffe6c0 1 -- 10.20.193.27:0/3358083149 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7c00a660 con 0x7fea80112fc0 2026-02-13T02:22:27.557 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.552+0000 7f97c6bd86c0 1 -- 10.20.193.27:0/1687116863 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f97c0115440 msgr2=0x7f97c0175de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.557 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.552+0000 7f97c6bd86c0 1 --2- 10.20.193.27:0/1687116863 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f97c0115440 0x7f97c0175de0 secure :-1 s=READY pgs=294 gs=1 cs=0 l=1 c_cookie=18008211115137528246 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f97bc00aa70 tx=0x7f97bc02fde0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.558 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.552+0000 7fea85ffe6c0 1 --2- 10.20.193.27:0/3358083149 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fea80112fc0 0x7fea80128580 secure :-1 s=READY pgs=295 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fea7c0054b0 tx=0x7fea7c0054e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.558 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.553+0000 7fea6e7fc6c0 1 -- 10.20.193.27:0/3358083149 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fea7c030cd0 con 0x7fea80112fc0 2026-02-13T02:22:27.558 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.553+0000 7fea6e7fc6c0 1 -- 10.20.193.27:0/3358083149 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fea7c032e40 con 0x7fea80112fc0 2026-02-13T02:22:27.558 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.553+0000 7fea6e7fc6c0 1 -- 10.20.193.27:0/3358083149 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fea7c032840 con 0x7fea80112fc0 2026-02-13T02:22:27.558 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.553+0000 7f97c6bd86c0 1 -- 10.20.193.27:0/1687116863 shutdown_connections 2026-02-13T02:22:27.559 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.553+0000 7f97c6bd86c0 1 --2- 10.20.193.27:0/1687116863 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f97c0115440 0x7f97c0175de0 unknown :-1 s=CLOSED pgs=294 gs=1 cs=0 l=1 c_cookie=18008211115137528246 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.559 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.553+0000 7f97c6bd86c0 1 --2- 10.20.193.27:0/1687116863 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f97c010fc20 0x7f97c0114c60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.559 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.553+0000 7f97c6bd86c0 1 --2- 10.20.193.27:0/1687116863 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f97c010f230 0x7f97c010f600 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.559 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.553+0000 7f97c6bd86c0 1 -- 10.20.193.27:0/1687116863 >> 10.20.193.27:0/1687116863 conn(0x7f97c00ecb30 msgr2=0x7f97c010cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.559 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.553+0000 7f97c6bd86c0 1 -- 10.20.193.27:0/1687116863 shutdown_connections 2026-02-13T02:22:27.559 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.553+0000 7f97c6bd86c0 1 -- 10.20.193.27:0/1687116863 wait complete. 2026-02-13T02:22:27.560 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.553+0000 7fea87a8c6c0 1 -- 10.20.193.27:0/3358083149 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea80129180 con 0x7fea80112fc0 2026-02-13T02:22:27.560 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.553+0000 7fea87a8c6c0 1 -- 10.20.193.27:0/3358083149 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea80129400 con 0x7fea80112fc0 2026-02-13T02:22:27.561 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.553+0000 7f97c6bd86c0 1 Processor -- start 2026-02-13T02:22:27.561 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.553+0000 7f97c6bd86c0 1 -- start start 2026-02-13T02:22:27.561 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.554+0000 7f97c6bd86c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f97c010f230 0x7f97c011d200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.561 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.554+0000 7f97c6bd86c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f97c0115440 0x7f97c011d740 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.561 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.554+0000 7f97c6bd86c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f97c0122b00 0x7f97c0122fa0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.561 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.554+0000 7f97c6bd86c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f97c0123510 con 0x7f97c0115440 2026-02-13T02:22:27.561 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.554+0000 7f97c6bd86c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f97c0123680 con 0x7f97c010f230 2026-02-13T02:22:27.561 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.554+0000 7f97c6bd86c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f97c0123840 con 0x7f97c0122b00 2026-02-13T02:22:27.561 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.554+0000 7f97c49496c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f97c010f230 0x7f97c011d200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.562 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.554+0000 7fea87a8c6c0 1 -- 10.20.193.27:0/3358083149 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea80065120 con 0x7fea80112fc0 2026-02-13T02:22:27.563 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.554+0000 7f97c49496c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f97c010f230 0x7f97c011d200 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.27:49334/0 (socket says 10.20.193.27:49334) 2026-02-13T02:22:27.563 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.554+0000 7f97c49496c0 1 -- 10.20.193.27:0/204852222 learned_addr learned my addr 10.20.193.27:0/204852222 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:27.563 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.554+0000 7f97b7fff6c0 1 --2- 10.20.193.27:0/204852222 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f97c0115440 0x7f97c011d740 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.563 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.554+0000 7f97c49496c0 1 -- 10.20.193.27:0/204852222 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f97c0122b00 msgr2=0x7f97c0122fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.563 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.554+0000 7f97c49496c0 1 --2- 10.20.193.27:0/204852222 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f97c0122b00 0x7f97c0122fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.564 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.554+0000 7f97c49496c0 1 -- 10.20.193.27:0/204852222 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f97c0115440 msgr2=0x7f97c011d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.564 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.554+0000 7f97c49496c0 1 --2- 10.20.193.27:0/204852222 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f97c0115440 0x7f97c011d740 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.564 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.554+0000 7f97c49496c0 1 -- 10.20.193.27:0/204852222 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a80094c0 con 0x7f97c010f230 2026-02-13T02:22:27.564 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.554+0000 7f97b7fff6c0 1 --2- 10.20.193.27:0/204852222 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f97c0115440 0x7f97c011d740 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:22:27.564 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.554+0000 7f97c49496c0 1 --2- 10.20.193.27:0/204852222 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f97c010f230 0x7f97c011d200 secure :-1 s=READY pgs=54 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f97b00092e0 tx=0x7f97b000bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.564 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.554+0000 7f97b5ffb6c0 1 -- 10.20.193.27:0/204852222 <== mon.2 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f97b0004250 con 0x7f97c010f230 2026-02-13T02:22:27.564 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.554+0000 7f97b5ffb6c0 1 -- 10.20.193.27:0/204852222 <== mon.2 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f97b00043b0 con 0x7f97c010f230 2026-02-13T02:22:27.564 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.554+0000 7f97c6bd86c0 1 -- 10.20.193.27:0/204852222 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97bc00a660 con 0x7f97c010f230 2026-02-13T02:22:27.564 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.555+0000 7f97b5ffb6c0 1 -- 10.20.193.27:0/204852222 <== mon.2 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f97b000cd40 con 0x7f97c010f230 2026-02-13T02:22:27.564 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.555+0000 7f97c6bd86c0 1 -- 10.20.193.27:0/204852222 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97c0164040 con 0x7f97c010f230 2026-02-13T02:22:27.565 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.555+0000 7fea6e7fc6c0 1 -- 10.20.193.27:0/3358083149 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7fea7c003810 con 0x7fea80112fc0 2026-02-13T02:22:27.565 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.555+0000 7f97c6bd86c0 1 -- 10.20.193.27:0/204852222 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97c0110500 con 0x7f97c010f230 2026-02-13T02:22:27.566 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.556+0000 7fea6e7fc6c0 1 --2- 10.20.193.27:0/3358083149 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fea5c0b5710 0x7fea5c0d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.566 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.556+0000 7fea6e7fc6c0 1 -- 10.20.193.27:0/3358083149 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7fea7c108e60 con 0x7fea80112fc0 2026-02-13T02:22:27.566 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.556+0000 7fea857fd6c0 1 --2- 10.20.193.27:0/3358083149 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fea5c0b5710 0x7fea5c0d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.566 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.556+0000 7fea857fd6c0 1 --2- 10.20.193.27:0/3358083149 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fea5c0b5710 0x7fea5c0d5bc0 secure :-1 s=READY pgs=206 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fea70005fd0 tx=0x7fea700093f0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.567 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149714 got 111669149713 for osd.4 2026-02-13T02:22:27.568 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.557+0000 7f97b5ffb6c0 1 -- 10.20.193.27:0/204852222 <== mon.2 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f97b00026b0 con 0x7f97c010f230 2026-02-13T02:22:27.568 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.557+0000 7f97b5ffb6c0 1 --2- 10.20.193.27:0/204852222 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f979c0b5710 0x7f979c0d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.568 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.558+0000 7f97b7fff6c0 1 --2- 10.20.193.27:0/204852222 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f979c0b5710 0x7f979c0d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.568 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.558+0000 7f97b5ffb6c0 1 -- 10.20.193.27:0/204852222 <== mon.2 v2:10.20.193.28:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f97b00e3a70 con 0x7f97c010f230 2026-02-13T02:22:27.568 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.558+0000 7fea6e7fc6c0 1 -- 10.20.193.27:0/3358083149 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fea7c0cb8c0 con 0x7fea80112fc0 2026-02-13T02:22:27.568 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.558+0000 7f97b5ffb6c0 1 -- 10.20.193.27:0/204852222 <== mon.2 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f97b00a94a0 con 0x7f97c010f230 2026-02-13T02:22:27.569 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.558+0000 7f97b7fff6c0 1 --2- 10.20.193.27:0/204852222 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f979c0b5710 0x7f979c0d5bc0 secure :-1 s=READY pgs=207 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f97c0123b90 tx=0x7f97a8009260 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.582 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:27.588 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874247 got 279172874246 for osd.11 2026-02-13T02:22:27.597 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378644 got 81604378643 for osd.3 2026-02-13T02:22:27.599 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691853 got 171798691852 for osd.7 2026-02-13T02:22:27.603 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462924 got 201863462923 for osd.8 2026-02-13T02:22:27.608 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.606+0000 7f8332f016c0 1 -- 10.20.193.27:0/2663354391 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f832c10f600 con 0x7f832c10f230 2026-02-13T02:22:27.608 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.607+0000 7f8321ffb6c0 1 -- 10.20.193.27:0/2663354391 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f831c0a98e0 con 0x7f832c10f230 2026-02-13T02:22:27.609 INFO:teuthology.orchestra.run.trial027.stdout:128849018896 2026-02-13T02:22:27.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.609+0000 7f8332f016c0 1 -- 10.20.193.27:0/2663354391 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f83080b5690 msgr2=0x7f83080d5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.609+0000 7f8332f016c0 1 --2- 10.20.193.27:0/2663354391 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f83080b5690 0x7f83080d5b40 secure :-1 s=READY pgs=204 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f83140092e0 tx=0x7f8314005b00 comp rx=0 tx=0).stop 2026-02-13T02:22:27.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.609+0000 7f8332f016c0 1 -- 10.20.193.27:0/2663354391 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f832c10f230 msgr2=0x7f832c11d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.609+0000 7f8332f016c0 1 --2- 10.20.193.27:0/2663354391 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f832c10f230 0x7f832c11d2f0 secure :-1 s=READY pgs=290 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f831c00ba30 tx=0x7f831c00bf00 comp rx=0 tx=0).stop 2026-02-13T02:22:27.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.609+0000 7f8332f016c0 1 -- 10.20.193.27:0/2663354391 shutdown_connections 2026-02-13T02:22:27.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.609+0000 7f8332f016c0 1 --2- 10.20.193.27:0/2663354391 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f83080b5690 0x7f83080d5b40 unknown :-1 s=CLOSED pgs=204 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.612 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.609+0000 7f8332f016c0 1 --2- 10.20.193.27:0/2663354391 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f832c122bf0 0x7f832c123090 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.612 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.609+0000 7f8332f016c0 1 --2- 10.20.193.27:0/2663354391 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f832c115440 0x7f832c11d830 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.612 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.609+0000 7f8332f016c0 1 --2- 10.20.193.27:0/2663354391 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f832c10f230 0x7f832c11d2f0 unknown :-1 s=CLOSED pgs=290 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.612 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.609+0000 7f8332f016c0 1 -- 10.20.193.27:0/2663354391 >> 10.20.193.27:0/2663354391 conn(0x7f832c0ecb30 msgr2=0x7f832c10cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.612 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.609+0000 7f8332f016c0 1 -- 10.20.193.27:0/2663354391 shutdown_connections 2026-02-13T02:22:27.612 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.609+0000 7f8332f016c0 1 -- 10.20.193.27:0/2663354391 wait complete. 2026-02-13T02:22:27.618 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.617+0000 7f40614d16c0 1 -- 10.20.193.27:0/1465769140 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f405c10f600 con 0x7f405c0f5270 2026-02-13T02:22:27.618 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.617+0000 7f403bfff6c0 1 -- 10.20.193.27:0/1465769140 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f404c0a7560 con 0x7f405c0f5270 2026-02-13T02:22:27.619 INFO:teuthology.orchestra.run.trial027.stdout:231928233994 2026-02-13T02:22:27.621 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.619+0000 7f40614d16c0 1 -- 10.20.193.27:0/1465769140 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f40300b5690 msgr2=0x7f40300d5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.621 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.619+0000 7f40614d16c0 1 --2- 10.20.193.27:0/1465769140 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f40300b5690 0x7f40300d5b40 secure :-1 s=READY pgs=205 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4050004750 tx=0x7f4050009260 comp rx=0 tx=0).stop 2026-02-13T02:22:27.621 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.619+0000 7f40614d16c0 1 -- 10.20.193.27:0/1465769140 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f405c0f5270 msgr2=0x7f405c0fa540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.621 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.619+0000 7f40614d16c0 1 --2- 10.20.193.27:0/1465769140 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f405c0f5270 0x7f405c0fa540 secure :-1 s=READY pgs=292 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f404c00a820 tx=0x7f404c00ce50 comp rx=0 tx=0).stop 2026-02-13T02:22:27.621 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.619+0000 7f40614d16c0 1 -- 10.20.193.27:0/1465769140 shutdown_connections 2026-02-13T02:22:27.621 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.619+0000 7f40614d16c0 1 --2- 10.20.193.27:0/1465769140 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f40300b5690 0x7f40300d5b40 unknown :-1 s=CLOSED pgs=205 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.622 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.619+0000 7f40614d16c0 1 --2- 10.20.193.27:0/1465769140 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f405c0f5270 0x7f405c0fa540 unknown :-1 s=CLOSED pgs=292 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.622 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.619+0000 7f40614d16c0 1 --2- 10.20.193.27:0/1465769140 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f405c115440 0x7f405c175ce0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.622 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.619+0000 7f40614d16c0 1 --2- 10.20.193.27:0/1465769140 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f405c10f230 0x7f405c0f4c40 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.622 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.619+0000 7f40614d16c0 1 -- 10.20.193.27:0/1465769140 >> 10.20.193.27:0/1465769140 conn(0x7f405c0ecb30 msgr2=0x7f405c10d890 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.622 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.619+0000 7f40614d16c0 1 -- 10.20.193.27:0/1465769140 shutdown_connections 2026-02-13T02:22:27.622 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.619+0000 7f40614d16c0 1 -- 10.20.193.27:0/1465769140 wait complete. 2026-02-13T02:22:27.661 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928233995 got 231928233994 for osd.9 2026-02-13T02:22:27.662 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.660+0000 7fea87a8c6c0 1 -- 10.20.193.27:0/3358083149 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fea8010c920 con 0x7fea80112fc0 2026-02-13T02:22:27.662 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.660+0000 7fea6e7fc6c0 1 -- 10.20.193.27:0/3358083149 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fea7c0cea70 con 0x7fea80112fc0 2026-02-13T02:22:27.662 INFO:teuthology.orchestra.run.trial027.stdout:42949672982 2026-02-13T02:22:27.664 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.663+0000 7fea87a8c6c0 1 -- 10.20.193.27:0/3358083149 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fea5c0b5710 msgr2=0x7fea5c0d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.664 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.663+0000 7fea87a8c6c0 1 --2- 10.20.193.27:0/3358083149 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fea5c0b5710 0x7fea5c0d5bc0 secure :-1 s=READY pgs=206 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fea70005fd0 tx=0x7fea700093f0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.665 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.663+0000 7fea87a8c6c0 1 -- 10.20.193.27:0/3358083149 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fea80112fc0 msgr2=0x7fea80128580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.665 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.663+0000 7fea87a8c6c0 1 --2- 10.20.193.27:0/3358083149 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fea80112fc0 0x7fea80128580 secure :-1 s=READY pgs=295 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fea7c0054b0 tx=0x7fea7c0054e0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.665 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.663+0000 7fea87a8c6c0 1 -- 10.20.193.27:0/3358083149 shutdown_connections 2026-02-13T02:22:27.665 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.663+0000 7fea87a8c6c0 1 --2- 10.20.193.27:0/3358083149 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fea5c0b5710 0x7fea5c0d5bc0 unknown :-1 s=CLOSED pgs=206 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.665 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.663+0000 7fea87a8c6c0 1 --2- 10.20.193.27:0/3358083149 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fea80112fc0 0x7fea80128580 unknown :-1 s=CLOSED pgs=295 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.665 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.663+0000 7fea87a8c6c0 1 --2- 10.20.193.27:0/3358083149 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fea80112b10 0x7fea801231c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.665 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.663+0000 7fea87a8c6c0 1 --2- 10.20.193.27:0/3358083149 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fea8010c4e0 0x7fea80122c80 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.665 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.663+0000 7fea87a8c6c0 1 -- 10.20.193.27:0/3358083149 >> 10.20.193.27:0/3358083149 conn(0x7fea80063af0 msgr2=0x7fea8010e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.665 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.663+0000 7fea87a8c6c0 1 -- 10.20.193.27:0/3358083149 shutdown_connections 2026-02-13T02:22:27.665 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.663+0000 7fea87a8c6c0 1 -- 10.20.193.27:0/3358083149 wait complete. 2026-02-13T02:22:27.670 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018897 got 128849018896 for osd.5 2026-02-13T02:22:27.670 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.669+0000 7f97c6bd86c0 1 -- 10.20.193.27:0/204852222 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f97c010f600 con 0x7f97c010f230 2026-02-13T02:22:27.670 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.669+0000 7f97b5ffb6c0 1 -- 10.20.193.27:0/204852222 <== mon.2 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f97b00a9680 con 0x7f97c010f230 2026-02-13T02:22:27.671 INFO:teuthology.orchestra.run.trial027.stdout:158913789966 2026-02-13T02:22:27.672 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.671+0000 7f97c6bd86c0 1 -- 10.20.193.27:0/204852222 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f979c0b5710 msgr2=0x7f979c0d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.673 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.671+0000 7f97c6bd86c0 1 --2- 10.20.193.27:0/204852222 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f979c0b5710 0x7f979c0d5bc0 secure :-1 s=READY pgs=207 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f97c0123b90 tx=0x7f97a8009260 comp rx=0 tx=0).stop 2026-02-13T02:22:27.673 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.671+0000 7f97c6bd86c0 1 -- 10.20.193.27:0/204852222 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f97c010f230 msgr2=0x7f97c011d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.673 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.671+0000 7f97c6bd86c0 1 --2- 10.20.193.27:0/204852222 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f97c010f230 0x7f97c011d200 secure :-1 s=READY pgs=54 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f97b00092e0 tx=0x7f97b000bc50 comp rx=0 tx=0).stop 2026-02-13T02:22:27.673 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.671+0000 7f97c6bd86c0 1 -- 10.20.193.27:0/204852222 shutdown_connections 2026-02-13T02:22:27.673 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.671+0000 7f97c6bd86c0 1 --2- 10.20.193.27:0/204852222 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f979c0b5710 0x7f979c0d5bc0 unknown :-1 s=CLOSED pgs=207 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.673 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.671+0000 7f97c6bd86c0 1 --2- 10.20.193.27:0/204852222 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f97c0122b00 0x7f97c0122fa0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.673 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.671+0000 7f97c6bd86c0 1 --2- 10.20.193.27:0/204852222 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f97c0115440 0x7f97c011d740 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.673 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.671+0000 7f97c6bd86c0 1 --2- 10.20.193.27:0/204852222 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f97c010f230 0x7f97c011d200 unknown :-1 s=CLOSED pgs=54 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.673 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.671+0000 7f97c6bd86c0 1 -- 10.20.193.27:0/204852222 >> 10.20.193.27:0/204852222 conn(0x7f97c00ecb30 msgr2=0x7f97c010cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.673 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.671+0000 7f97c6bd86c0 1 -- 10.20.193.27:0/204852222 shutdown_connections 2026-02-13T02:22:27.673 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.671+0000 7f97c6bd86c0 1 -- 10.20.193.27:0/204852222 wait complete. 2026-02-13T02:22:27.705 INFO:tasks.cephadm.ceph_manager.ceph:need seq 42949672983 got 42949672982 for osd.1 2026-02-13T02:22:27.714 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.711+0000 7f98524ef6c0 1 -- 10.20.193.27:0/1170615700 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f984c10fc20 msgr2=0x7f984c114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.715 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.711+0000 7f98524ef6c0 1 --2- 10.20.193.27:0/1170615700 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f984c10fc20 0x7f984c114c60 secure :-1 s=READY pgs=296 gs=1 cs=0 l=1 c_cookie=5355356781657592870 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f983c0099a0 tx=0x7f983c02ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:27.715 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.711+0000 7f98524ef6c0 1 -- 10.20.193.27:0/1170615700 shutdown_connections 2026-02-13T02:22:27.715 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.711+0000 7f98524ef6c0 1 --2- 10.20.193.27:0/1170615700 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f984c115440 0x7f984c175de0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.715 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.711+0000 7f98524ef6c0 1 --2- 10.20.193.27:0/1170615700 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f984c10fc20 0x7f984c114c60 unknown :-1 s=CLOSED pgs=296 gs=1 cs=0 l=1 c_cookie=5355356781657592870 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.715 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.711+0000 7f98524ef6c0 1 --2- 10.20.193.27:0/1170615700 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f984c10f230 0x7f984c10f600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.715 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.711+0000 7f98524ef6c0 1 -- 10.20.193.27:0/1170615700 >> 10.20.193.27:0/1170615700 conn(0x7f984c0ecb30 msgr2=0x7f984c10cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.715 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.711+0000 7f98524ef6c0 1 -- 10.20.193.27:0/1170615700 shutdown_connections 2026-02-13T02:22:27.715 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.711+0000 7f98524ef6c0 1 -- 10.20.193.27:0/1170615700 wait complete. 2026-02-13T02:22:27.715 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.711+0000 7f98524ef6c0 1 Processor -- start 2026-02-13T02:22:27.715 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.711+0000 7f98524ef6c0 1 -- start start 2026-02-13T02:22:27.716 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.711+0000 7f98524ef6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f984c10f230 0x7f984c0f4cd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.716 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.712+0000 7f98524ef6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f984c115440 0x7f984c175ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.716 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.712+0000 7f98524ef6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f984c0f5300 0x7f984c0fa5d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.716 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.712+0000 7f98524ef6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f984c0f59f0 con 0x7f984c115440 2026-02-13T02:22:27.716 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.712+0000 7f98524ef6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f984c0fab10 con 0x7f984c10f230 2026-02-13T02:22:27.716 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.712+0000 7f98524ef6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f984c0facd0 con 0x7f984c0f5300 2026-02-13T02:22:27.716 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.712+0000 7f984b7fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f984c115440 0x7f984c175ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.716 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.712+0000 7f984b7fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f984c115440 0x7f984c175ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:53624/0 (socket says 10.20.193.27:53624) 2026-02-13T02:22:27.716 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.712+0000 7f984b7fe6c0 1 -- 10.20.193.27:0/3980158692 learned_addr learned my addr 10.20.193.27:0/3980158692 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:27.716 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.712+0000 7f9850a616c0 1 --2- 10.20.193.27:0/3980158692 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f984c0f5300 0x7f984c0fa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.716 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.712+0000 7f984bfff6c0 1 --2- 10.20.193.27:0/3980158692 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f984c10f230 0x7f984c0f4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.717 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.712+0000 7f984b7fe6c0 1 -- 10.20.193.27:0/3980158692 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f984c0f5300 msgr2=0x7f984c0fa5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.717 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.712+0000 7f984b7fe6c0 1 --2- 10.20.193.27:0/3980158692 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f984c0f5300 0x7f984c0fa5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.717 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.712+0000 7f984b7fe6c0 1 -- 10.20.193.27:0/3980158692 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f984c10f230 msgr2=0x7f984c0f4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.717 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.712+0000 7f984b7fe6c0 1 --2- 10.20.193.27:0/3980158692 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f984c10f230 0x7f984c0f4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.717 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.712+0000 7f984b7fe6c0 1 -- 10.20.193.27:0/3980158692 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f983c009590 con 0x7f984c115440 2026-02-13T02:22:27.717 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.712+0000 7f9850a616c0 1 --2- 10.20.193.27:0/3980158692 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f984c0f5300 0x7f984c0fa5d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:22:27.717 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.712+0000 7f984b7fe6c0 1 --2- 10.20.193.27:0/3980158692 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f984c115440 0x7f984c175ce0 secure :-1 s=READY pgs=297 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f983c0023c0 tx=0x7f983c02fbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.717 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.712+0000 7f98497fa6c0 1 -- 10.20.193.27:0/3980158692 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f983c002ae0 con 0x7f984c115440 2026-02-13T02:22:27.717 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.712+0000 7f98497fa6c0 1 -- 10.20.193.27:0/3980158692 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f983c002c40 con 0x7f984c115440 2026-02-13T02:22:27.717 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.712+0000 7f98524ef6c0 1 -- 10.20.193.27:0/3980158692 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f984c0faf50 con 0x7f984c115440 2026-02-13T02:22:27.717 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.712+0000 7f98497fa6c0 1 -- 10.20.193.27:0/3980158692 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f983c0372f0 con 0x7f984c115440 2026-02-13T02:22:27.717 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.712+0000 7f98524ef6c0 1 -- 10.20.193.27:0/3980158692 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f984c0fb420 con 0x7f984c115440 2026-02-13T02:22:27.717 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.712+0000 7f98524ef6c0 1 -- 10.20.193.27:0/3980158692 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f984c110500 con 0x7f984c115440 2026-02-13T02:22:27.719 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.714+0000 7f98497fa6c0 1 -- 10.20.193.27:0/3980158692 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f983c007810 con 0x7f984c115440 2026-02-13T02:22:27.719 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.715+0000 7f98497fa6c0 1 --2- 10.20.193.27:0/3980158692 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f98200b5710 0x7f98200d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:27.719 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.715+0000 7f984bfff6c0 1 --2- 10.20.193.27:0/3980158692 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f98200b5710 0x7f98200d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:27.719 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.715+0000 7f98497fa6c0 1 -- 10.20.193.27:0/3980158692 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f983c107bc0 con 0x7f984c115440 2026-02-13T02:22:27.719 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.715+0000 7f98497fa6c0 1 -- 10.20.193.27:0/3980158692 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f983c0ce080 con 0x7f984c115440 2026-02-13T02:22:27.719 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.715+0000 7f984bfff6c0 1 --2- 10.20.193.27:0/3980158692 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f98200b5710 0x7f98200d5bc0 secure :-1 s=READY pgs=208 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9838005980 tx=0x7f9838005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:27.720 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913789967 got 158913789966 for osd.6 2026-02-13T02:22:27.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.805+0000 7f98524ef6c0 1 -- 10.20.193.27:0/3980158692 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f984c10fc20 con 0x7f984c115440 2026-02-13T02:22:27.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.806+0000 7f98497fa6c0 1 -- 10.20.193.27:0/3980158692 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f983c007e60 con 0x7f984c115440 2026-02-13T02:22:27.807 INFO:teuthology.orchestra.run.trial027.stdout:249108103176 2026-02-13T02:22:27.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.806+0000 7f98524ef6c0 1 -- 10.20.193.27:0/3980158692 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f98200b5710 msgr2=0x7f98200d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.806+0000 7f98524ef6c0 1 --2- 10.20.193.27:0/3980158692 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f98200b5710 0x7f98200d5bc0 secure :-1 s=READY pgs=208 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9838005980 tx=0x7f9838005910 comp rx=0 tx=0).stop 2026-02-13T02:22:27.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.806+0000 7f98524ef6c0 1 -- 10.20.193.27:0/3980158692 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f984c115440 msgr2=0x7f984c175ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:27.809 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.806+0000 7f98524ef6c0 1 --2- 10.20.193.27:0/3980158692 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f984c115440 0x7f984c175ce0 secure :-1 s=READY pgs=297 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f983c0023c0 tx=0x7f983c02fbf0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.809 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.807+0000 7f98524ef6c0 1 -- 10.20.193.27:0/3980158692 shutdown_connections 2026-02-13T02:22:27.809 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.807+0000 7f98524ef6c0 1 --2- 10.20.193.27:0/3980158692 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f98200b5710 0x7f98200d5bc0 unknown :-1 s=CLOSED pgs=208 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.809 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.807+0000 7f98524ef6c0 1 --2- 10.20.193.27:0/3980158692 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f984c0f5300 0x7f984c0fa5d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.809 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.807+0000 7f98524ef6c0 1 --2- 10.20.193.27:0/3980158692 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f984c115440 0x7f984c175ce0 unknown :-1 s=CLOSED pgs=297 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.809 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.807+0000 7f98524ef6c0 1 --2- 10.20.193.27:0/3980158692 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f984c10f230 0x7f984c0f4cd0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:27.809 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.807+0000 7f98524ef6c0 1 -- 10.20.193.27:0/3980158692 >> 10.20.193.27:0/3980158692 conn(0x7f984c0ecb30 msgr2=0x7f984c113da0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:27.809 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.807+0000 7f98524ef6c0 1 -- 10.20.193.27:0/3980158692 shutdown_connections 2026-02-13T02:22:27.809 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:27.807+0000 7f98524ef6c0 1 -- 10.20.193.27:0/3980158692 wait complete. 2026-02-13T02:22:27.847 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103177 got 249108103176 for osd.10 2026-02-13T02:22:28.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:28 trial027 ceph-mon[24567]: pgmap v189: 97 pgs: 97 active+clean; 629 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 6.1 KiB/s rd, 8.0 KiB/s wr, 8 op/s 2026-02-13T02:22:28.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:28 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2538192014' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2026-02-13T02:22:28.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:28 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/787528539' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2026-02-13T02:22:28.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:28 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/779774875' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2026-02-13T02:22:28.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:28 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/483797881' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2026-02-13T02:22:28.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:28 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/807215352' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2026-02-13T02:22:28.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:28 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/846292312' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2026-02-13T02:22:28.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:28 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2669206531' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2026-02-13T02:22:28.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:28 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2663354391' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2026-02-13T02:22:28.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:28 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1465769140' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2026-02-13T02:22:28.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:28 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3358083149' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2026-02-13T02:22:28.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:28 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/204852222' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2026-02-13T02:22:28.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:28 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3980158692' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2026-02-13T02:22:28.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:28 trial028 ceph-mon[26701]: pgmap v189: 97 pgs: 97 active+clean; 629 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 6.1 KiB/s rd, 8.0 KiB/s wr, 8 op/s 2026-02-13T02:22:28.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:28 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2538192014' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2026-02-13T02:22:28.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:28 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/787528539' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2026-02-13T02:22:28.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:28 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/779774875' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2026-02-13T02:22:28.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:28 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/483797881' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2026-02-13T02:22:28.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:28 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/807215352' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2026-02-13T02:22:28.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:28 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/846292312' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2026-02-13T02:22:28.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:28 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2669206531' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2026-02-13T02:22:28.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:28 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2663354391' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2026-02-13T02:22:28.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:28 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1465769140' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2026-02-13T02:22:28.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:28 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3358083149' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2026-02-13T02:22:28.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:28 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/204852222' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2026-02-13T02:22:28.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:28 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3980158692' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2026-02-13T02:22:28.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:28 trial107 ceph-mon[27422]: pgmap v189: 97 pgs: 97 active+clean; 629 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 6.1 KiB/s rd, 8.0 KiB/s wr, 8 op/s 2026-02-13T02:22:28.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:28 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2538192014' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2026-02-13T02:22:28.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:28 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/787528539' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2026-02-13T02:22:28.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:28 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/779774875' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2026-02-13T02:22:28.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:28 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/483797881' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2026-02-13T02:22:28.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:28 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/807215352' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2026-02-13T02:22:28.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:28 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/846292312' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2026-02-13T02:22:28.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:28 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2669206531' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2026-02-13T02:22:28.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:28 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2663354391' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2026-02-13T02:22:28.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:28 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1465769140' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2026-02-13T02:22:28.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:28 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3358083149' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2026-02-13T02:22:28.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:28 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/204852222' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2026-02-13T02:22:28.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:28 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3980158692' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2026-02-13T02:22:28.545 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.2 2026-02-13T02:22:28.547 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.0 2026-02-13T02:22:28.568 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.4 2026-02-13T02:22:28.589 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.11 2026-02-13T02:22:28.598 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.3 2026-02-13T02:22:28.600 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.7 2026-02-13T02:22:28.604 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.8 2026-02-13T02:22:28.662 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.9 2026-02-13T02:22:28.665 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:28.666 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:28.671 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.5 2026-02-13T02:22:28.706 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.1 2026-02-13T02:22:28.712 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:28.713 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:28.721 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.6 2026-02-13T02:22:28.760 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:28.761 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:28.768 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:28.802 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:28.813 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.809+0000 7fa510c1b6c0 1 -- 10.20.193.27:0/2408730144 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa50c10fc20 msgr2=0x7fa50c114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.813 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.810+0000 7fa510c1b6c0 1 --2- 10.20.193.27:0/2408730144 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa50c10fc20 0x7fa50c114c60 secure :-1 s=READY pgs=298 gs=1 cs=0 l=1 c_cookie=1335798374501527934 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa4f40099a0 tx=0x7fa4f402ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:28.813 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.810+0000 7fa510c1b6c0 1 -- 10.20.193.27:0/2408730144 shutdown_connections 2026-02-13T02:22:28.813 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.810+0000 7fa510c1b6c0 1 --2- 10.20.193.27:0/2408730144 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa50c115440 0x7fa50c175de0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.814 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.810+0000 7fa510c1b6c0 1 --2- 10.20.193.27:0/2408730144 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa50c10fc20 0x7fa50c114c60 unknown :-1 s=CLOSED pgs=298 gs=1 cs=0 l=1 c_cookie=1335798374501527934 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.814 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.810+0000 7fa510c1b6c0 1 --2- 10.20.193.27:0/2408730144 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa50c10f230 0x7fa50c10f600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.814 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.810+0000 7fa510c1b6c0 1 -- 10.20.193.27:0/2408730144 >> 10.20.193.27:0/2408730144 conn(0x7fa50c0ecb30 msgr2=0x7fa50c10cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:28.814 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.810+0000 7fa510c1b6c0 1 -- 10.20.193.27:0/2408730144 shutdown_connections 2026-02-13T02:22:28.814 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.810+0000 7fa510c1b6c0 1 -- 10.20.193.27:0/2408730144 wait complete. 2026-02-13T02:22:28.814 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.810+0000 7fa510c1b6c0 1 Processor -- start 2026-02-13T02:22:28.814 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.810+0000 7fa510c1b6c0 1 -- start start 2026-02-13T02:22:28.814 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.810+0000 7fa510c1b6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa50c10f230 0x7fa50c0f4cd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.814 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.810+0000 7fa510c1b6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa50c115440 0x7fa50c175ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.814 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.810+0000 7fa510c1b6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa50c0f5300 0x7fa50c0fa5d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.815 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.810+0000 7fa510c1b6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c0f59f0 con 0x7fa50c10f230 2026-02-13T02:22:28.815 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.810+0000 7fa510c1b6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c0fab10 con 0x7fa50c115440 2026-02-13T02:22:28.815 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.810+0000 7fa510c1b6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c0facd0 con 0x7fa50c0f5300 2026-02-13T02:22:28.815 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.810+0000 7fa50a5756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa50c10f230 0x7fa50c0f4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.815 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.810+0000 7fa50a5756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa50c10f230 0x7fa50c0f4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:53640/0 (socket says 10.20.193.27:53640) 2026-02-13T02:22:28.815 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.810+0000 7fa50a5756c0 1 -- 10.20.193.27:0/1542532650 learned_addr learned my addr 10.20.193.27:0/1542532650 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:28.815 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.810+0000 7fa509d746c0 1 --2- 10.20.193.27:0/1542532650 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa50c115440 0x7fa50c175ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.815 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.810+0000 7fa50a5756c0 1 -- 10.20.193.27:0/1542532650 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa50c0f5300 msgr2=0x7fa50c0fa5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.815 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.810+0000 7fa50ad766c0 1 --2- 10.20.193.27:0/1542532650 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa50c0f5300 0x7fa50c0fa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.815 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.810+0000 7fa50a5756c0 1 --2- 10.20.193.27:0/1542532650 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa50c0f5300 0x7fa50c0fa5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.815 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.810+0000 7fa50a5756c0 1 -- 10.20.193.27:0/1542532650 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa50c115440 msgr2=0x7fa50c175ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.816 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.810+0000 7fa50a5756c0 1 --2- 10.20.193.27:0/1542532650 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa50c115440 0x7fa50c175ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.816 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.810+0000 7fa50a5756c0 1 -- 10.20.193.27:0/1542532650 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f4009590 con 0x7fa50c10f230 2026-02-13T02:22:28.816 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.810+0000 7fa509d746c0 1 --2- 10.20.193.27:0/1542532650 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa50c115440 0x7fa50c175ce0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:22:28.816 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.810+0000 7fa50ad766c0 1 --2- 10.20.193.27:0/1542532650 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa50c0f5300 0x7fa50c0fa5d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:22:28.816 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.811+0000 7fa50a5756c0 1 --2- 10.20.193.27:0/1542532650 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa50c10f230 0x7fa50c0f4cd0 secure :-1 s=READY pgs=300 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa500009490 tx=0x7fa50000bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:28.816 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.811+0000 7fa4f37fe6c0 1 -- 10.20.193.27:0/1542532650 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa500004a80 con 0x7fa50c10f230 2026-02-13T02:22:28.816 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.811+0000 7fa4f37fe6c0 1 -- 10.20.193.27:0/1542532650 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa500009030 con 0x7fa50c10f230 2026-02-13T02:22:28.816 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.811+0000 7fa510c1b6c0 1 -- 10.20.193.27:0/1542532650 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa50c0fafb0 con 0x7fa50c10f230 2026-02-13T02:22:28.816 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.811+0000 7fa4f37fe6c0 1 -- 10.20.193.27:0/1542532650 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa50000cdd0 con 0x7fa50c10f230 2026-02-13T02:22:28.816 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.811+0000 7fa510c1b6c0 1 -- 10.20.193.27:0/1542532650 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa50c0fb440 con 0x7fa50c10f230 2026-02-13T02:22:28.817 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.811+0000 7fe8770476c0 1 -- 10.20.193.27:0/1945312882 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe870115410 msgr2=0x7fe870175db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.817 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.811+0000 7fe8770476c0 1 --2- 10.20.193.27:0/1945312882 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe870115410 0x7fe870175db0 secure :-1 s=READY pgs=299 gs=1 cs=0 l=1 c_cookie=1786067912680274336 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe86c00aa70 tx=0x7fe86c02fde0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.817 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.811+0000 7fe8770476c0 1 -- 10.20.193.27:0/1945312882 shutdown_connections 2026-02-13T02:22:28.817 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.811+0000 7fe8770476c0 1 --2- 10.20.193.27:0/1945312882 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe870115410 0x7fe870175db0 unknown :-1 s=CLOSED pgs=299 gs=1 cs=0 l=1 c_cookie=1786067912680274336 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.817 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.811+0000 7fe8770476c0 1 --2- 10.20.193.27:0/1945312882 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe87010fbf0 0x7fe870114c30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.817 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.811+0000 7fe8770476c0 1 --2- 10.20.193.27:0/1945312882 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe87010f200 0x7fe87010f5d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.817 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.811+0000 7fe8770476c0 1 -- 10.20.193.27:0/1945312882 >> 10.20.193.27:0/1945312882 conn(0x7fe8700ecb00 msgr2=0x7fe87010cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:28.817 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.811+0000 7fe8770476c0 1 -- 10.20.193.27:0/1945312882 shutdown_connections 2026-02-13T02:22:28.817 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.811+0000 7fe8770476c0 1 -- 10.20.193.27:0/1945312882 wait complete. 2026-02-13T02:22:28.818 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.811+0000 7fa510c1b6c0 1 -- 10.20.193.27:0/1542532650 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa50c110500 con 0x7fa50c10f230 2026-02-13T02:22:28.819 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.811+0000 7fe8770476c0 1 Processor -- start 2026-02-13T02:22:28.819 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.812+0000 7fe8770476c0 1 -- start start 2026-02-13T02:22:28.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.812+0000 7fe8770476c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe87010f200 0x7fe87011d200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.812+0000 7fe8770476c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe870115410 0x7fe87011d740 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.812+0000 7fe8770476c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe870122b00 0x7fe870122fa0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.812+0000 7fe8770476c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe870123510 con 0x7fe870122b00 2026-02-13T02:22:28.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.812+0000 7fe8770476c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe870123680 con 0x7fe87010f200 2026-02-13T02:22:28.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.812+0000 7fe8770476c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe870123840 con 0x7fe870115410 2026-02-13T02:22:28.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.812+0000 7fe8755b96c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe870122b00 0x7fe870122fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.812+0000 7fe8755b96c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe870122b00 0x7fe870122fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:53656/0 (socket says 10.20.193.27:53656) 2026-02-13T02:22:28.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.812+0000 7fe8755b96c0 1 -- 10.20.193.27:0/2104342111 learned_addr learned my addr 10.20.193.27:0/2104342111 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:28.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.812+0000 7fe867fff6c0 1 --2- 10.20.193.27:0/2104342111 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe870115410 0x7fe87011d740 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.812+0000 7fe874db86c0 1 --2- 10.20.193.27:0/2104342111 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe87010f200 0x7fe87011d200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.821 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.812+0000 7fe8755b96c0 1 -- 10.20.193.27:0/2104342111 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe870115410 msgr2=0x7fe87011d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.821 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.812+0000 7fe8755b96c0 1 --2- 10.20.193.27:0/2104342111 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe870115410 0x7fe87011d740 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.821 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.812+0000 7fe8755b96c0 1 -- 10.20.193.27:0/2104342111 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe87010f200 msgr2=0x7fe87011d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.821 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.812+0000 7fe8755b96c0 1 --2- 10.20.193.27:0/2104342111 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe87010f200 0x7fe87011d200 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.821 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.812+0000 7fe8755b96c0 1 -- 10.20.193.27:0/2104342111 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe86c00a660 con 0x7fe870122b00 2026-02-13T02:22:28.821 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.812+0000 7fe8755b96c0 1 --2- 10.20.193.27:0/2104342111 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe870122b00 0x7fe870122fa0 secure :-1 s=READY pgs=301 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe86c006ec0 tx=0x7fe86c005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:28.822 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.813+0000 7fa4f37fe6c0 1 -- 10.20.193.27:0/1542532650 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7fa500004f30 con 0x7fa50c10f230 2026-02-13T02:22:28.822 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.813+0000 7fa4f37fe6c0 1 --2- 10.20.193.27:0/1542532650 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa4e40b5690 0x7fa4e40d5b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.822 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.814+0000 7fa509d746c0 1 --2- 10.20.193.27:0/1542532650 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa4e40b5690 0x7fa4e40d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.822 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.814+0000 7fa4f37fe6c0 1 -- 10.20.193.27:0/1542532650 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7fa5000e39d0 con 0x7fa50c10f230 2026-02-13T02:22:28.822 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.814+0000 7fa509d746c0 1 --2- 10.20.193.27:0/1542532650 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa4e40b5690 0x7fa4e40d5b40 secure :-1 s=READY pgs=209 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa4f40023e0 tx=0x7fa4f4005b00 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:28.822 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.814+0000 7fa4f37fe6c0 1 -- 10.20.193.27:0/1542532650 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fa5000a6430 con 0x7fa50c10f230 2026-02-13T02:22:28.823 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.814+0000 7fe865ffb6c0 1 -- 10.20.193.27:0/2104342111 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe86c003c40 con 0x7fe870122b00 2026-02-13T02:22:28.823 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.814+0000 7fe865ffb6c0 1 -- 10.20.193.27:0/2104342111 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fe86c005780 con 0x7fe870122b00 2026-02-13T02:22:28.823 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.814+0000 7fe8770476c0 1 -- 10.20.193.27:0/2104342111 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe870163c90 con 0x7fe870122b00 2026-02-13T02:22:28.824 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.814+0000 7fe865ffb6c0 1 -- 10.20.193.27:0/2104342111 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe86c0382f0 con 0x7fe870122b00 2026-02-13T02:22:28.824 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.814+0000 7fe8770476c0 1 -- 10.20.193.27:0/2104342111 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8701640c0 con 0x7fe870122b00 2026-02-13T02:22:28.824 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.815+0000 7fe8770476c0 1 -- 10.20.193.27:0/2104342111 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8701104d0 con 0x7fe870122b00 2026-02-13T02:22:28.824 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.816+0000 7fe865ffb6c0 1 -- 10.20.193.27:0/2104342111 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7fe86c008260 con 0x7fe870122b00 2026-02-13T02:22:28.824 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.817+0000 7fe865ffb6c0 1 --2- 10.20.193.27:0/2104342111 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe84c0b5710 0x7fe84c0d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.824 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.817+0000 7fe874db86c0 1 --2- 10.20.193.27:0/2104342111 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe84c0b5710 0x7fe84c0d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.824 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.817+0000 7fe865ffb6c0 1 -- 10.20.193.27:0/2104342111 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7fe86c108f70 con 0x7fe870122b00 2026-02-13T02:22:28.824 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.817+0000 7fe874db86c0 1 --2- 10.20.193.27:0/2104342111 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe84c0b5710 0x7fe84c0d5bc0 secure :-1 s=READY pgs=210 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe860005980 tx=0x7fe860005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:28.824 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.819+0000 7fe865ffb6c0 1 -- 10.20.193.27:0/2104342111 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fe86c0cb990 con 0x7fe870122b00 2026-02-13T02:22:28.843 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:28.846 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:28.848 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.10 2026-02-13T02:22:28.852 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.850+0000 7ff24a1846c0 1 -- 10.20.193.27:0/3821592965 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff24410cfb0 msgr2=0x7ff24410d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.852 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.850+0000 7ff24a1846c0 1 --2- 10.20.193.27:0/3821592965 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff24410cfb0 0x7ff24410d380 secure :-1 s=READY pgs=55 gs=2 cs=0 l=1 c_cookie=7146786627156748244 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff2300098a0 tx=0x7ff23002ec40 comp rx=0 tx=0).stop 2026-02-13T02:22:28.852 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.850+0000 7fa6030a36c0 1 -- 10.20.193.27:0/3038979277 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa5fc060740 msgr2=0x7fa5fc060b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.853 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.850+0000 7fa6030a36c0 1 --2- 10.20.193.27:0/3038979277 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa5fc060740 0x7fa5fc060b10 secure :-1 s=READY pgs=302 gs=1 cs=0 l=1 c_cookie=16244813939535896921 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa5ec009970 tx=0x7fa5ec02ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:28.853 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.850+0000 7fa6030a36c0 1 -- 10.20.193.27:0/3038979277 shutdown_connections 2026-02-13T02:22:28.853 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.850+0000 7fa6030a36c0 1 --2- 10.20.193.27:0/3038979277 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa5fc1556c0 0x7fa5fc175aa0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.853 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.850+0000 7fa6030a36c0 1 --2- 10.20.193.27:0/3038979277 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa5fc061050 0x7fa5fc155180 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.853 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.850+0000 7fa6030a36c0 1 --2- 10.20.193.27:0/3038979277 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa5fc060740 0x7fa5fc060b10 unknown :-1 s=CLOSED pgs=302 gs=1 cs=0 l=1 c_cookie=16244813939535896921 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.853 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.850+0000 7fa6030a36c0 1 -- 10.20.193.27:0/3038979277 >> 10.20.193.27:0/3038979277 conn(0x7fa5fc0ea8b0 msgr2=0x7fa5fc10acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:28.853 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.850+0000 7fa6030a36c0 1 -- 10.20.193.27:0/3038979277 shutdown_connections 2026-02-13T02:22:28.853 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.850+0000 7fa6030a36c0 1 -- 10.20.193.27:0/3038979277 wait complete. 2026-02-13T02:22:28.853 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.851+0000 7fa6030a36c0 1 Processor -- start 2026-02-13T02:22:28.853 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.851+0000 7fa6030a36c0 1 -- start start 2026-02-13T02:22:28.854 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.850+0000 7ff24a1846c0 1 -- 10.20.193.27:0/3821592965 shutdown_connections 2026-02-13T02:22:28.854 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.850+0000 7ff24a1846c0 1 --2- 10.20.193.27:0/3821592965 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff2441131c0 0x7ff244173b60 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.854 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.850+0000 7ff24a1846c0 1 --2- 10.20.193.27:0/3821592965 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff24410d9a0 0x7ff2441129e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.854 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.850+0000 7ff24a1846c0 1 --2- 10.20.193.27:0/3821592965 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff24410cfb0 0x7ff24410d380 unknown :-1 s=CLOSED pgs=55 gs=2 cs=0 l=1 c_cookie=7146786627156748244 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.854 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.850+0000 7ff24a1846c0 1 -- 10.20.193.27:0/3821592965 >> 10.20.193.27:0/3821592965 conn(0x7ff2440ea8b0 msgr2=0x7ff24410acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:28.855 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.850+0000 7ff24a1846c0 1 -- 10.20.193.27:0/3821592965 shutdown_connections 2026-02-13T02:22:28.855 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.850+0000 7ff24a1846c0 1 -- 10.20.193.27:0/3821592965 wait complete. 2026-02-13T02:22:28.855 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.851+0000 7ff24a1846c0 1 Processor -- start 2026-02-13T02:22:28.855 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.851+0000 7ff24a1846c0 1 -- start start 2026-02-13T02:22:28.855 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.852+0000 7ff24a1846c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff24410cfb0 0x7ff2440f4be0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.855 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.852+0000 7ff24a1846c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff2441131c0 0x7ff244173a60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.855 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.852+0000 7ff24a1846c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff2440f5210 0x7ff2440fa4e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.855 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.851+0000 7fa6030a36c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa5fc060740 0x7fa5fc161530 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.855 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.851+0000 7fa6030a36c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa5fc061050 0x7fa5fc161a70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.856 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.851+0000 7fa6030a36c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa5fc1556c0 0x7fa5fc166e30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.856 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.851+0000 7fa6030a36c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa5fc167480 con 0x7fa5fc061050 2026-02-13T02:22:28.856 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.851+0000 7fa6030a36c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa5fc1675f0 con 0x7fa5fc1556c0 2026-02-13T02:22:28.856 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.851+0000 7fa6030a36c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa5fc1677b0 con 0x7fa5fc060740 2026-02-13T02:22:28.856 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.851+0000 7fa5f3fff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa5fc061050 0x7fa5fc161a70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.856 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.851+0000 7fa5f3fff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa5fc061050 0x7fa5fc161a70 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:53686/0 (socket says 10.20.193.27:53686) 2026-02-13T02:22:28.856 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.851+0000 7fa5f3fff6c0 1 -- 10.20.193.27:0/2594122028 learned_addr learned my addr 10.20.193.27:0/2594122028 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:28.857 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.852+0000 7ff2437fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff24410cfb0 0x7ff2440f4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.857 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.852+0000 7ff24a1846c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff2440f5900 con 0x7ff24410cfb0 2026-02-13T02:22:28.857 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.851+0000 7fa5f3fff6c0 1 -- 10.20.193.27:0/2594122028 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa5fc060740 msgr2=0x7fa5fc161530 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2026-02-13T02:22:28.857 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.851+0000 7fa5f3fff6c0 1 --2- 10.20.193.27:0/2594122028 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa5fc060740 0x7fa5fc161530 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.857 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.851+0000 7fa5f3fff6c0 1 -- 10.20.193.27:0/2594122028 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa5fc1556c0 msgr2=0x7fa5fc166e30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2026-02-13T02:22:28.857 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.851+0000 7fa5f3fff6c0 1 --2- 10.20.193.27:0/2594122028 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa5fc1556c0 0x7fa5fc166e30 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.857 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.851+0000 7fa5f3fff6c0 1 -- 10.20.193.27:0/2594122028 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ec009590 con 0x7fa5fc061050 2026-02-13T02:22:28.857 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.851+0000 7fa5f3fff6c0 1 --2- 10.20.193.27:0/2594122028 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa5fc061050 0x7fa5fc161a70 secure :-1 s=READY pgs=303 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa5e400b6c0 tx=0x7fa5e400bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:28.858 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.852+0000 7fa5f1ffb6c0 1 -- 10.20.193.27:0/2594122028 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa5e4004a80 con 0x7fa5fc061050 2026-02-13T02:22:28.858 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.852+0000 7fa5f1ffb6c0 1 -- 10.20.193.27:0/2594122028 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa5e4009030 con 0x7fa5fc061050 2026-02-13T02:22:28.858 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.852+0000 7fa6030a36c0 1 -- 10.20.193.27:0/2594122028 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5fc167a90 con 0x7fa5fc061050 2026-02-13T02:22:28.858 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.852+0000 7fa5f1ffb6c0 1 -- 10.20.193.27:0/2594122028 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa5e400cc10 con 0x7fa5fc061050 2026-02-13T02:22:28.858 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.852+0000 7fa6030a36c0 1 -- 10.20.193.27:0/2594122028 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5fc10cdc0 con 0x7fa5fc061050 2026-02-13T02:22:28.859 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.852+0000 7ff24a1846c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff2440faa20 con 0x7ff2440f5210 2026-02-13T02:22:28.859 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.852+0000 7ff24a1846c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff2440fabe0 con 0x7ff2441131c0 2026-02-13T02:22:28.859 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.852+0000 7ff2437fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff24410cfb0 0x7ff2440f4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:53692/0 (socket says 10.20.193.27:53692) 2026-02-13T02:22:28.859 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.852+0000 7ff2437fe6c0 1 -- 10.20.193.27:0/2227139836 learned_addr learned my addr 10.20.193.27:0/2227139836 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:28.859 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.852+0000 7ff243fff6c0 1 --2- 10.20.193.27:0/2227139836 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff2440f5210 0x7ff2440fa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.859 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.852+0000 7ff242ffd6c0 1 --2- 10.20.193.27:0/2227139836 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff2441131c0 0x7ff244173a60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.859 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.852+0000 7ff2437fe6c0 1 -- 10.20.193.27:0/2227139836 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff2441131c0 msgr2=0x7ff244173a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.859 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.852+0000 7ff2437fe6c0 1 --2- 10.20.193.27:0/2227139836 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff2441131c0 0x7ff244173a60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.859 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.852+0000 7ff2437fe6c0 1 -- 10.20.193.27:0/2227139836 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff2440f5210 msgr2=0x7ff2440fa4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.859 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.852+0000 7ff2437fe6c0 1 --2- 10.20.193.27:0/2227139836 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff2440f5210 0x7ff2440fa4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.859 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.852+0000 7ff2437fe6c0 1 -- 10.20.193.27:0/2227139836 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2300094c0 con 0x7ff24410cfb0 2026-02-13T02:22:28.860 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.852+0000 7ff242ffd6c0 1 --2- 10.20.193.27:0/2227139836 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff2441131c0 0x7ff244173a60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:22:28.860 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.852+0000 7fa5cb7fe6c0 1 -- 10.20.193.27:0/2594122028 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5fc061c50 con 0x7fa5fc061050 2026-02-13T02:22:28.861 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.852+0000 7ff2437fe6c0 1 --2- 10.20.193.27:0/2227139836 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff24410cfb0 0x7ff2440f4be0 secure :-1 s=READY pgs=304 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff2300097f0 tx=0x7ff23002fad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:28.861 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.853+0000 7ff240ff96c0 1 -- 10.20.193.27:0/2227139836 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff230002ae0 con 0x7ff24410cfb0 2026-02-13T02:22:28.861 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.853+0000 7ff24a1846c0 1 -- 10.20.193.27:0/2227139836 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2440fae60 con 0x7ff24410cfb0 2026-02-13T02:22:28.861 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.853+0000 7ff24a1846c0 1 -- 10.20.193.27:0/2227139836 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2440fb330 con 0x7ff24410cfb0 2026-02-13T02:22:28.861 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.853+0000 7ff24a1846c0 1 -- 10.20.193.27:0/2227139836 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff24410e280 con 0x7ff24410cfb0 2026-02-13T02:22:28.861 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.855+0000 7fa5f1ffb6c0 1 -- 10.20.193.27:0/2594122028 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7fa5e4005000 con 0x7fa5fc061050 2026-02-13T02:22:28.862 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.855+0000 7ff240ff96c0 1 -- 10.20.193.27:0/2227139836 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff230002c40 con 0x7ff24410cfb0 2026-02-13T02:22:28.862 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.855+0000 7fa5f1ffb6c0 1 --2- 10.20.193.27:0/2594122028 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa5d80be120 0x7fa5d80de5d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.862 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.856+0000 7ff240ff96c0 1 -- 10.20.193.27:0/2227139836 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff230007e90 con 0x7ff24410cfb0 2026-02-13T02:22:28.863 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.856+0000 7fa600e146c0 1 --2- 10.20.193.27:0/2594122028 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa5d80be120 0x7fa5d80de5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.863 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.856+0000 7fa5f1ffb6c0 1 -- 10.20.193.27:0/2594122028 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7fa5e40e3950 con 0x7fa5fc061050 2026-02-13T02:22:28.863 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.856+0000 7fa5f1ffb6c0 1 -- 10.20.193.27:0/2594122028 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fa5e40a9380 con 0x7fa5fc061050 2026-02-13T02:22:28.863 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.856+0000 7fa600e146c0 1 --2- 10.20.193.27:0/2594122028 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa5d80be120 0x7fa5d80de5d0 secure :-1 s=READY pgs=211 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa5ec009970 tx=0x7fa5ec0023a0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:28.864 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.857+0000 7ff240ff96c0 1 -- 10.20.193.27:0/2227139836 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7ff2300071c0 con 0x7ff24410cfb0 2026-02-13T02:22:28.864 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.858+0000 7ff240ff96c0 1 --2- 10.20.193.27:0/2227139836 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff2180b5690 0x7ff2180d5b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.864 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.859+0000 7ff242ffd6c0 1 --2- 10.20.193.27:0/2227139836 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff2180b5690 0x7ff2180d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.864 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.859+0000 7ff240ff96c0 1 -- 10.20.193.27:0/2227139836 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7ff230107c30 con 0x7ff24410cfb0 2026-02-13T02:22:28.864 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.859+0000 7ff240ff96c0 1 -- 10.20.193.27:0/2227139836 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7ff2301405e0 con 0x7ff24410cfb0 2026-02-13T02:22:28.865 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:28.865 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.859+0000 7ff242ffd6c0 1 --2- 10.20.193.27:0/2227139836 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff2180b5690 0x7ff2180d5b40 secure :-1 s=READY pgs=212 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff2440fb570 tx=0x7ff234009260 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:28.906 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.904+0000 7f66200406c0 1 -- 10.20.193.27:0/1380348118 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f6618115440 msgr2=0x7f6618175de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.906 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.904+0000 7f66200406c0 1 --2- 10.20.193.27:0/1380348118 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f6618115440 0x7f6618175de0 secure :-1 s=READY pgs=305 gs=1 cs=0 l=1 c_cookie=8533382151324167174 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f661400aa70 tx=0x7f661402fde0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.907 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.905+0000 7f66200406c0 1 -- 10.20.193.27:0/1380348118 shutdown_connections 2026-02-13T02:22:28.907 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.905+0000 7f66200406c0 1 --2- 10.20.193.27:0/1380348118 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f6618115440 0x7f6618175de0 unknown :-1 s=CLOSED pgs=305 gs=1 cs=0 l=1 c_cookie=8533382151324167174 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.907 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.905+0000 7f66200406c0 1 --2- 10.20.193.27:0/1380348118 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f661810fc20 0x7f6618114c60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.907 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.905+0000 7f66200406c0 1 --2- 10.20.193.27:0/1380348118 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f661810f230 0x7f661810f600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.907 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.905+0000 7f66200406c0 1 -- 10.20.193.27:0/1380348118 >> 10.20.193.27:0/1380348118 conn(0x7f66180ecb30 msgr2=0x7f661810cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:28.907 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.905+0000 7f66200406c0 1 -- 10.20.193.27:0/1380348118 shutdown_connections 2026-02-13T02:22:28.907 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.905+0000 7f66200406c0 1 -- 10.20.193.27:0/1380348118 wait complete. 2026-02-13T02:22:28.908 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.905+0000 7fa334da86c0 1 -- 10.20.193.27:0/2393327024 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa33010f210 msgr2=0x7fa33010f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.908 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.905+0000 7fa334da86c0 1 --2- 10.20.193.27:0/2393327024 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa33010f210 0x7fa33010f5e0 secure :-1 s=READY pgs=306 gs=1 cs=0 l=1 c_cookie=2696153972887207122 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa3240099a0 tx=0x7fa32402ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:28.908 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.905+0000 7fa334da86c0 1 -- 10.20.193.27:0/2393327024 shutdown_connections 2026-02-13T02:22:28.908 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.905+0000 7fa334da86c0 1 --2- 10.20.193.27:0/2393327024 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa330115420 0x7fa330175dc0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.908 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.905+0000 7fa334da86c0 1 --2- 10.20.193.27:0/2393327024 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa33010fc00 0x7fa330114c40 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.908 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.905+0000 7fa334da86c0 1 --2- 10.20.193.27:0/2393327024 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa33010f210 0x7fa33010f5e0 unknown :-1 s=CLOSED pgs=306 gs=1 cs=0 l=1 c_cookie=2696153972887207122 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.908 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.905+0000 7fa334da86c0 1 -- 10.20.193.27:0/2393327024 >> 10.20.193.27:0/2393327024 conn(0x7fa3300ecb30 msgr2=0x7fa33010cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:28.909 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.905+0000 7f66200406c0 1 Processor -- start 2026-02-13T02:22:28.909 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.905+0000 7f66200406c0 1 -- start start 2026-02-13T02:22:28.909 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.905+0000 7fa334da86c0 1 -- 10.20.193.27:0/2393327024 shutdown_connections 2026-02-13T02:22:28.910 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.905+0000 7fa334da86c0 1 -- 10.20.193.27:0/2393327024 wait complete. 2026-02-13T02:22:28.910 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.905+0000 7f66200406c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f661810f230 0x7f661811d200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.910 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7f3dd88276c0 1 -- 10.20.193.27:0/365567332 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3dd010fbf0 msgr2=0x7f3dd0114c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.911 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7f3dd88276c0 1 --2- 10.20.193.27:0/365567332 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3dd010fbf0 0x7f3dd0114c30 secure :-1 s=READY pgs=57 gs=2 cs=0 l=1 c_cookie=3723531712475698133 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3dbc013460 tx=0x7f3dbc02ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:28.911 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7f3dd88276c0 1 -- 10.20.193.27:0/365567332 shutdown_connections 2026-02-13T02:22:28.911 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7f3dd88276c0 1 --2- 10.20.193.27:0/365567332 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3dd0115410 0x7f3dd0175db0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.911 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7f3dd88276c0 1 --2- 10.20.193.27:0/365567332 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3dd010fbf0 0x7f3dd0114c30 unknown :-1 s=CLOSED pgs=57 gs=2 cs=0 l=1 c_cookie=3723531712475698133 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.911 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7f3dd88276c0 1 --2- 10.20.193.27:0/365567332 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3dd010f200 0x7f3dd010f5d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.911 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7f3dd88276c0 1 -- 10.20.193.27:0/365567332 >> 10.20.193.27:0/365567332 conn(0x7f3dd00ecb00 msgr2=0x7f3dd010cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:28.911 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7fa334da86c0 1 Processor -- start 2026-02-13T02:22:28.912 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7fa334da86c0 1 -- start start 2026-02-13T02:22:28.912 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7f3dd88276c0 1 -- 10.20.193.27:0/365567332 shutdown_connections 2026-02-13T02:22:28.912 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7fa334da86c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa33010f210 0x7fa33011d260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.913 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7fa334da86c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa330115420 0x7fa33011d7a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.913 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.905+0000 7f661ddb56c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f661810f230 0x7f661811d200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.913 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.905+0000 7f661ddb56c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f661810f230 0x7f661811d200 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:53730/0 (socket says 10.20.193.27:53730) 2026-02-13T02:22:28.914 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7f661ddb56c0 1 -- 10.20.193.27:0/1952203966 learned_addr learned my addr 10.20.193.27:0/1952203966 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:28.914 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7f66200406c0 1 --2- 10.20.193.27:0/1952203966 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f6618115440 0x7f661811d740 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.914 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7f66200406c0 1 --2- 10.20.193.27:0/1952203966 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6618122b00 0x7f6618122fa0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.914 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7f66200406c0 1 -- 10.20.193.27:0/1952203966 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6618123510 con 0x7f661810f230 2026-02-13T02:22:28.914 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7f66200406c0 1 -- 10.20.193.27:0/1952203966 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6618123680 con 0x7f6618122b00 2026-02-13T02:22:28.914 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7f66200406c0 1 -- 10.20.193.27:0/1952203966 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6618123840 con 0x7f6618115440 2026-02-13T02:22:28.914 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7f661d5b46c0 1 --2- 10.20.193.27:0/1952203966 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f6618115440 0x7f661811d740 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.914 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7f661e5b66c0 1 --2- 10.20.193.27:0/1952203966 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6618122b00 0x7f6618122fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.914 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7f661ddb56c0 1 -- 10.20.193.27:0/1952203966 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f6618115440 msgr2=0x7f661811d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.914 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7f661ddb56c0 1 --2- 10.20.193.27:0/1952203966 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f6618115440 0x7f661811d740 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.914 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7f661ddb56c0 1 -- 10.20.193.27:0/1952203966 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6618122b00 msgr2=0x7f6618122fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.915 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7fa334da86c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa330122b60 0x7fa330123000 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.915 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7f661ddb56c0 1 --2- 10.20.193.27:0/1952203966 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6618122b00 0x7f6618122fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.915 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7f661ddb56c0 1 -- 10.20.193.27:0/1952203966 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661400a660 con 0x7f661810f230 2026-02-13T02:22:28.915 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7f661e5b66c0 1 --2- 10.20.193.27:0/1952203966 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6618122b00 0x7f6618122fa0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:22:28.916 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7fa334da86c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa330123540 con 0x7fa330122b60 2026-02-13T02:22:28.916 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7fa334da86c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa3301236b0 con 0x7fa33010f210 2026-02-13T02:22:28.916 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7fa334da86c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa330123870 con 0x7fa330115420 2026-02-13T02:22:28.917 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7fa32e5756c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa33010f210 0x7fa33011d260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.917 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7fa32e5756c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa33010f210 0x7fa33011d260 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.27:49466/0 (socket says 10.20.193.27:49466) 2026-02-13T02:22:28.917 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7fa32e5756c0 1 -- 10.20.193.27:0/1981214267 learned_addr learned my addr 10.20.193.27:0/1981214267 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:28.917 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7fa32ed766c0 1 --2- 10.20.193.27:0/1981214267 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa330122b60 0x7fa330123000 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.917 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7fa32dd746c0 1 --2- 10.20.193.27:0/1981214267 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa330115420 0x7fa33011d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.917 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7fa32e5756c0 1 -- 10.20.193.27:0/1981214267 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa330115420 msgr2=0x7fa33011d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.917 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7fa32e5756c0 1 --2- 10.20.193.27:0/1981214267 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa330115420 0x7fa33011d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.917 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7fa32e5756c0 1 -- 10.20.193.27:0/1981214267 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa330122b60 msgr2=0x7fa330123000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.917 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7f3dd88276c0 1 -- 10.20.193.27:0/365567332 wait complete. 2026-02-13T02:22:28.918 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7f3dd88276c0 1 Processor -- start 2026-02-13T02:22:28.918 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7f3dd88276c0 1 -- start start 2026-02-13T02:22:28.918 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.906+0000 7fa32e5756c0 1 --2- 10.20.193.27:0/1981214267 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa330122b60 0x7fa330123000 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.918 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.907+0000 7fa32e5756c0 1 -- 10.20.193.27:0/1981214267 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa324009590 con 0x7fa33010f210 2026-02-13T02:22:28.918 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.907+0000 7fa32ed766c0 1 --2- 10.20.193.27:0/1981214267 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa330122b60 0x7fa330123000 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-13T02:22:28.919 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.907+0000 7f661ddb56c0 1 --2- 10.20.193.27:0/1952203966 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f661810f230 0x7f661811d200 secure :-1 s=READY pgs=307 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f660800b710 tx=0x7f660800bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:28.919 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.907+0000 7f3dd88276c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3dd010f200 0x7f3dd00f41e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.919 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.907+0000 7f3dd88276c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3dd0115410 0x7f3dd0175cb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.919 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.907+0000 7f3dd88276c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3dd00f4810 0x7f3dd00f9ae0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.920 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.907+0000 7f3dd88276c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd00f4f00 con 0x7f3dd00f4810 2026-02-13T02:22:28.920 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.907+0000 7f3dd88276c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd00fa020 con 0x7f3dd0115410 2026-02-13T02:22:28.920 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.907+0000 7fa32e5756c0 1 --2- 10.20.193.27:0/1981214267 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa33010f210 0x7fa33011d260 secure :-1 s=READY pgs=56 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa32402f230 tx=0x7fa32402fd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:28.921 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.907+0000 7f3dd88276c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd00fa1e0 con 0x7f3dd010f200 2026-02-13T02:22:28.921 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.907+0000 7f3dd6d996c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3dd00f4810 0x7f3dd00f9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.921 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.907+0000 7f3dd6d996c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3dd00f4810 0x7f3dd00f9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:53758/0 (socket says 10.20.193.27:53758) 2026-02-13T02:22:28.921 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.907+0000 7f3dd6d996c0 1 -- 10.20.193.27:0/742511478 learned_addr learned my addr 10.20.193.27:0/742511478 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:28.921 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.907+0000 7f3dd65986c0 1 --2- 10.20.193.27:0/742511478 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3dd010f200 0x7f3dd00f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.921 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.907+0000 7f3dd5d976c0 1 --2- 10.20.193.27:0/742511478 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3dd0115410 0x7f3dd0175cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.922 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.907+0000 7fa3177fe6c0 1 -- 10.20.193.27:0/1981214267 <== mon.2 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa324002ba0 con 0x7fa33010f210 2026-02-13T02:22:28.922 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.907+0000 7fa3177fe6c0 1 -- 10.20.193.27:0/1981214267 <== mon.2 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa324002d00 con 0x7fa33010f210 2026-02-13T02:22:28.922 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.907+0000 7fa334da86c0 1 -- 10.20.193.27:0/1981214267 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa330163cf0 con 0x7fa33010f210 2026-02-13T02:22:28.922 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.907+0000 7fa3177fe6c0 1 -- 10.20.193.27:0/1981214267 <== mon.2 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa3240372f0 con 0x7fa33010f210 2026-02-13T02:22:28.922 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.907+0000 7f3dd6d996c0 1 -- 10.20.193.27:0/742511478 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3dd010f200 msgr2=0x7f3dd00f41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.922 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.907+0000 7f3dd6d996c0 1 --2- 10.20.193.27:0/742511478 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3dd010f200 0x7f3dd00f41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.923 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.907+0000 7f3dd6d996c0 1 -- 10.20.193.27:0/742511478 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3dd0115410 msgr2=0x7f3dd0175cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.923 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.907+0000 7f3dd6d996c0 1 --2- 10.20.193.27:0/742511478 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3dd0115410 0x7f3dd0175cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.923 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.907+0000 7f3dd6d996c0 1 -- 10.20.193.27:0/742511478 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc801c520 con 0x7f3dd00f4810 2026-02-13T02:22:28.923 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.907+0000 7fa334da86c0 1 -- 10.20.193.27:0/1981214267 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3301641c0 con 0x7fa33010f210 2026-02-13T02:22:28.923 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.907+0000 7f6606ffd6c0 1 -- 10.20.193.27:0/1952203966 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6608004250 con 0x7f661810f230 2026-02-13T02:22:28.924 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.907+0000 7f66200406c0 1 -- 10.20.193.27:0/1952203966 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6618163cf0 con 0x7f661810f230 2026-02-13T02:22:28.924 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.907+0000 7f6606ffd6c0 1 -- 10.20.193.27:0/1952203966 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6608009070 con 0x7f661810f230 2026-02-13T02:22:28.924 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.907+0000 7f6606ffd6c0 1 -- 10.20.193.27:0/1952203966 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f660800ccd0 con 0x7f661810f230 2026-02-13T02:22:28.924 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.907+0000 7f66200406c0 1 -- 10.20.193.27:0/1952203966 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66181641d0 con 0x7f661810f230 2026-02-13T02:22:28.924 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.907+0000 7f3dd6d996c0 1 --2- 10.20.193.27:0/742511478 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3dd00f4810 0x7f3dd00f9ae0 secure :-1 s=READY pgs=308 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3dc4006ea0 tx=0x7f3dc400ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:28.924 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.908+0000 7fa3157fa6c0 1 -- 10.20.193.27:0/1981214267 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2f8005310 con 0x7fa33010f210 2026-02-13T02:22:28.925 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.908+0000 7f3dbb7fe6c0 1 -- 10.20.193.27:0/742511478 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3dc4003c40 con 0x7f3dd00f4810 2026-02-13T02:22:28.925 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.908+0000 7f3dbb7fe6c0 1 -- 10.20.193.27:0/742511478 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3dc4003da0 con 0x7f3dd00f4810 2026-02-13T02:22:28.926 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.908+0000 7f3dd88276c0 1 -- 10.20.193.27:0/742511478 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dbc013050 con 0x7f3dd00f4810 2026-02-13T02:22:28.926 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.908+0000 7f3dd88276c0 1 -- 10.20.193.27:0/742511478 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dd00fa7c0 con 0x7f3dd00f4810 2026-02-13T02:22:28.926 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.908+0000 7f3dbb7fe6c0 1 -- 10.20.193.27:0/742511478 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3dc40081c0 con 0x7f3dd00f4810 2026-02-13T02:22:28.926 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.909+0000 7f3dd88276c0 1 -- 10.20.193.27:0/742511478 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dd01104d0 con 0x7f3dd00f4810 2026-02-13T02:22:28.926 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.908+0000 7f66200406c0 1 -- 10.20.193.27:0/1952203966 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6618110500 con 0x7f661810f230 2026-02-13T02:22:28.926 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.909+0000 7f6606ffd6c0 1 -- 10.20.193.27:0/1952203966 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f66080028c0 con 0x7f661810f230 2026-02-13T02:22:28.927 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.910+0000 7f3dbb7fe6c0 1 -- 10.20.193.27:0/742511478 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f3dc400d590 con 0x7f3dd00f4810 2026-02-13T02:22:28.927 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.910+0000 7f6606ffd6c0 1 --2- 10.20.193.27:0/1952203966 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f65f40b5690 0x7f65f40d5b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.927 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.910+0000 7f6606ffd6c0 1 -- 10.20.193.27:0/1952203966 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f66080e3d60 con 0x7f661810f230 2026-02-13T02:22:28.927 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.910+0000 7fa3177fe6c0 1 -- 10.20.193.27:0/1981214267 <== mon.2 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7fa3240078d0 con 0x7fa33010f210 2026-02-13T02:22:28.928 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.910+0000 7f661d5b46c0 1 --2- 10.20.193.27:0/1952203966 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f65f40b5690 0x7f65f40d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.928 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.911+0000 7fa3177fe6c0 1 --2- 10.20.193.27:0/1981214267 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa3080b5710 0x7fa3080d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.928 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.911+0000 7f661d5b46c0 1 --2- 10.20.193.27:0/1952203966 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f65f40b5690 0x7f65f40d5b40 secure :-1 s=READY pgs=213 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f660c005980 tx=0x7f660c005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:28.929 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.911+0000 7f3dbb7fe6c0 1 --2- 10.20.193.27:0/742511478 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3dac0b5590 0x7f3dac0d5a40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.929 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.911+0000 7f3dd65986c0 1 --2- 10.20.193.27:0/742511478 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3dac0b5590 0x7f3dac0d5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.929 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.911+0000 7fa3177fe6c0 1 -- 10.20.193.27:0/1981214267 <== mon.2 v2:10.20.193.28:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7fa324107b80 con 0x7fa33010f210 2026-02-13T02:22:28.930 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.911+0000 7f3dbb7fe6c0 1 -- 10.20.193.27:0/742511478 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f3dc40e4ca0 con 0x7f3dd00f4810 2026-02-13T02:22:28.930 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.911+0000 7f3dd65986c0 1 --2- 10.20.193.27:0/742511478 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3dac0b5590 0x7f3dac0d5a40 secure :-1 s=READY pgs=214 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3dc8004750 tx=0x7f3dc8009260 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:28.931 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.911+0000 7fa32dd746c0 1 --2- 10.20.193.27:0/1981214267 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa3080b5710 0x7fa3080d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.931 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.911+0000 7fa3177fe6c0 1 -- 10.20.193.27:0/1981214267 <== mon.2 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fa3240ca630 con 0x7fa33010f210 2026-02-13T02:22:28.931 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.911+0000 7fa32dd746c0 1 --2- 10.20.193.27:0/1981214267 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa3080b5710 0x7fa3080d5bc0 secure :-1 s=READY pgs=215 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa318005980 tx=0x7fa318005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:28.931 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.912+0000 7f6606ffd6c0 1 -- 10.20.193.27:0/1952203966 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f66080a6780 con 0x7f661810f230 2026-02-13T02:22:28.931 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.913+0000 7f3dbb7fe6c0 1 -- 10.20.193.27:0/742511478 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f3dc40a76c0 con 0x7f3dd00f4810 2026-02-13T02:22:28.932 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.917+0000 7fa510c1b6c0 1 -- 10.20.193.27:0/1542532650 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fa50c10f600 con 0x7fa50c10f230 2026-02-13T02:22:28.932 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.918+0000 7fa4f37fe6c0 1 -- 10.20.193.27:0/1542532650 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa5000a95e0 con 0x7fa50c10f230 2026-02-13T02:22:28.932 INFO:teuthology.orchestra.run.trial027.stdout:55834574868 2026-02-13T02:22:28.933 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.920+0000 7fa510c1b6c0 1 -- 10.20.193.27:0/1542532650 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa4e40b5690 msgr2=0x7fa4e40d5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.933 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.920+0000 7fa510c1b6c0 1 --2- 10.20.193.27:0/1542532650 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa4e40b5690 0x7fa4e40d5b40 secure :-1 s=READY pgs=209 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa4f40023e0 tx=0x7fa4f4005b00 comp rx=0 tx=0).stop 2026-02-13T02:22:28.933 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.920+0000 7fa510c1b6c0 1 -- 10.20.193.27:0/1542532650 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa50c10f230 msgr2=0x7fa50c0f4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.933 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.920+0000 7fa510c1b6c0 1 --2- 10.20.193.27:0/1542532650 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa50c10f230 0x7fa50c0f4cd0 secure :-1 s=READY pgs=300 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa500009490 tx=0x7fa50000bc50 comp rx=0 tx=0).stop 2026-02-13T02:22:28.933 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.920+0000 7fa510c1b6c0 1 -- 10.20.193.27:0/1542532650 shutdown_connections 2026-02-13T02:22:28.933 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.920+0000 7fa510c1b6c0 1 --2- 10.20.193.27:0/1542532650 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa4e40b5690 0x7fa4e40d5b40 unknown :-1 s=CLOSED pgs=209 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.933 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.920+0000 7fa510c1b6c0 1 --2- 10.20.193.27:0/1542532650 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa50c0f5300 0x7fa50c0fa5d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.934 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.920+0000 7fa510c1b6c0 1 --2- 10.20.193.27:0/1542532650 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa50c115440 0x7fa50c175ce0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.934 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.920+0000 7fa510c1b6c0 1 --2- 10.20.193.27:0/1542532650 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa50c10f230 0x7fa50c0f4cd0 unknown :-1 s=CLOSED pgs=300 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.934 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.920+0000 7fa510c1b6c0 1 -- 10.20.193.27:0/1542532650 >> 10.20.193.27:0/1542532650 conn(0x7fa50c0ecb30 msgr2=0x7fa50c113da0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:28.934 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.920+0000 7fa510c1b6c0 1 -- 10.20.193.27:0/1542532650 shutdown_connections 2026-02-13T02:22:28.934 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.920+0000 7fa510c1b6c0 1 -- 10.20.193.27:0/1542532650 wait complete. 2026-02-13T02:22:28.934 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.926+0000 7fe8770476c0 1 -- 10.20.193.27:0/2104342111 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fe87010f5d0 con 0x7fe870122b00 2026-02-13T02:22:28.934 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.926+0000 7fe865ffb6c0 1 -- 10.20.193.27:0/2104342111 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe86c0ceb40 con 0x7fe870122b00 2026-02-13T02:22:28.935 INFO:teuthology.orchestra.run.trial027.stdout:30064771095 2026-02-13T02:22:28.937 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.927+0000 7fa2ffccc6c0 1 -- 10.20.193.27:0/3228942783 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa2f810f230 msgr2=0x7fa2f810f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.937 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.927+0000 7fa2ffccc6c0 1 --2- 10.20.193.27:0/3228942783 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa2f810f230 0x7fa2f810f600 secure :-1 s=READY pgs=57 gs=2 cs=0 l=1 c_cookie=9156426291524079098 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa2e8009970 tx=0x7fa2e802ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:28.937 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.927+0000 7fa2ffccc6c0 1 -- 10.20.193.27:0/3228942783 shutdown_connections 2026-02-13T02:22:28.937 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.927+0000 7fa2ffccc6c0 1 --2- 10.20.193.27:0/3228942783 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa2f8115440 0x7fa2f8175de0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.937 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.927+0000 7fa2ffccc6c0 1 --2- 10.20.193.27:0/3228942783 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa2f810fc20 0x7fa2f8114c60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.938 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.927+0000 7fa2ffccc6c0 1 --2- 10.20.193.27:0/3228942783 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa2f810f230 0x7fa2f810f600 unknown :-1 s=CLOSED pgs=57 gs=2 cs=0 l=1 c_cookie=9156426291524079098 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.938 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.927+0000 7fa2ffccc6c0 1 -- 10.20.193.27:0/3228942783 >> 10.20.193.27:0/3228942783 conn(0x7fa2f80ecb30 msgr2=0x7fa2f810cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:28.938 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.927+0000 7fa2ffccc6c0 1 -- 10.20.193.27:0/3228942783 shutdown_connections 2026-02-13T02:22:28.938 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.927+0000 7fa2ffccc6c0 1 -- 10.20.193.27:0/3228942783 wait complete. 2026-02-13T02:22:28.938 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.927+0000 7fa2ffccc6c0 1 Processor -- start 2026-02-13T02:22:28.938 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.927+0000 7fa2ffccc6c0 1 -- start start 2026-02-13T02:22:28.938 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.927+0000 7fa2ffccc6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa2f810f230 0x7fa2f811d220 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.938 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.927+0000 7fa2ffccc6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa2f8115440 0x7fa2f811d760 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.938 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.927+0000 7fa2ffccc6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa2f8122b20 0x7fa2f8122fc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.938 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.927+0000 7fa2ffccc6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f8123530 con 0x7fa2f8122b20 2026-02-13T02:22:28.938 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.928+0000 7fa2ffccc6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f81236a0 con 0x7fa2f810f230 2026-02-13T02:22:28.938 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.928+0000 7fa2ffccc6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f8123860 con 0x7fa2f8115440 2026-02-13T02:22:28.938 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.928+0000 7fa2fe2426c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa2f8122b20 0x7fa2f8122fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.938 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.928+0000 7fa2fe2426c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa2f8122b20 0x7fa2f8122fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:53764/0 (socket says 10.20.193.27:53764) 2026-02-13T02:22:28.938 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.928+0000 7fa2fe2426c0 1 -- 10.20.193.27:0/782565218 learned_addr learned my addr 10.20.193.27:0/782565218 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:28.939 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.928+0000 7fa2fd2406c0 1 --2- 10.20.193.27:0/782565218 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa2f8115440 0x7fa2f811d760 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.939 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.928+0000 7fa2fda416c0 1 --2- 10.20.193.27:0/782565218 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa2f810f230 0x7fa2f811d220 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.939 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.928+0000 7fa2fe2426c0 1 -- 10.20.193.27:0/782565218 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa2f8115440 msgr2=0x7fa2f811d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.939 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.928+0000 7fa2fe2426c0 1 --2- 10.20.193.27:0/782565218 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa2f8115440 0x7fa2f811d760 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.939 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.928+0000 7fa2fe2426c0 1 -- 10.20.193.27:0/782565218 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa2f810f230 msgr2=0x7fa2f811d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.939 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.928+0000 7fa2fe2426c0 1 --2- 10.20.193.27:0/782565218 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa2f810f230 0x7fa2f811d220 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.939 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.928+0000 7fa2fe2426c0 1 -- 10.20.193.27:0/782565218 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e8009590 con 0x7fa2f8122b20 2026-02-13T02:22:28.939 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.928+0000 7fa2fe2426c0 1 --2- 10.20.193.27:0/782565218 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa2f8122b20 0x7fa2f8122fc0 secure :-1 s=READY pgs=309 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa2f4006ea0 tx=0x7fa2f400ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:28.939 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.928+0000 7fa2e6ffd6c0 1 -- 10.20.193.27:0/782565218 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa2f4003c40 con 0x7fa2f8122b20 2026-02-13T02:22:28.939 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.928+0000 7fa2ffccc6c0 1 -- 10.20.193.27:0/782565218 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2f8163d10 con 0x7fa2f8122b20 2026-02-13T02:22:28.939 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.928+0000 7fa2e6ffd6c0 1 -- 10.20.193.27:0/782565218 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa2f4003da0 con 0x7fa2f8122b20 2026-02-13T02:22:28.939 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.928+0000 7fa2e6ffd6c0 1 -- 10.20.193.27:0/782565218 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa2f40081c0 con 0x7fa2f8122b20 2026-02-13T02:22:28.939 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.928+0000 7fa2ffccc6c0 1 -- 10.20.193.27:0/782565218 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2f81641f0 con 0x7fa2f8122b20 2026-02-13T02:22:28.939 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.929+0000 7fa2e4ff96c0 1 -- 10.20.193.27:0/782565218 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2c4005310 con 0x7fa2f8122b20 2026-02-13T02:22:28.940 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.929+0000 7fe8770476c0 1 -- 10.20.193.27:0/2104342111 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe84c0b5710 msgr2=0x7fe84c0d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.940 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.929+0000 7fe8770476c0 1 --2- 10.20.193.27:0/2104342111 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe84c0b5710 0x7fe84c0d5bc0 secure :-1 s=READY pgs=210 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe860005980 tx=0x7fe860005910 comp rx=0 tx=0).stop 2026-02-13T02:22:28.940 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.929+0000 7fe8770476c0 1 -- 10.20.193.27:0/2104342111 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe870122b00 msgr2=0x7fe870122fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.940 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.929+0000 7fe8770476c0 1 --2- 10.20.193.27:0/2104342111 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe870122b00 0x7fe870122fa0 secure :-1 s=READY pgs=301 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe86c006ec0 tx=0x7fe86c005520 comp rx=0 tx=0).stop 2026-02-13T02:22:28.940 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.930+0000 7fe8770476c0 1 -- 10.20.193.27:0/2104342111 shutdown_connections 2026-02-13T02:22:28.940 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.930+0000 7fe8770476c0 1 --2- 10.20.193.27:0/2104342111 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe84c0b5710 0x7fe84c0d5bc0 unknown :-1 s=CLOSED pgs=210 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.940 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.930+0000 7fe8770476c0 1 --2- 10.20.193.27:0/2104342111 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe870122b00 0x7fe870122fa0 unknown :-1 s=CLOSED pgs=301 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.940 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.930+0000 7fe8770476c0 1 --2- 10.20.193.27:0/2104342111 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe870115410 0x7fe87011d740 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.940 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.930+0000 7fe8770476c0 1 --2- 10.20.193.27:0/2104342111 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe87010f200 0x7fe87011d200 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.940 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.930+0000 7fe8770476c0 1 -- 10.20.193.27:0/2104342111 >> 10.20.193.27:0/2104342111 conn(0x7fe8700ecb00 msgr2=0x7fe87010cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:28.941 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.930+0000 7fe8770476c0 1 -- 10.20.193.27:0/2104342111 shutdown_connections 2026-02-13T02:22:28.941 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.930+0000 7fe8770476c0 1 -- 10.20.193.27:0/2104342111 wait complete. 2026-02-13T02:22:28.941 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.930+0000 7fa2e6ffd6c0 1 -- 10.20.193.27:0/782565218 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7fa2f4005860 con 0x7fa2f8122b20 2026-02-13T02:22:28.941 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.931+0000 7fa2e6ffd6c0 1 --2- 10.20.193.27:0/782565218 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa2d40b5710 0x7fa2d40d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.941 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.931+0000 7fa2fda416c0 1 --2- 10.20.193.27:0/782565218 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa2d40b5710 0x7fa2d40d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.941 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.931+0000 7fa2e6ffd6c0 1 -- 10.20.193.27:0/782565218 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7fa2f40e4af0 con 0x7fa2f8122b20 2026-02-13T02:22:28.942 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.931+0000 7fa2fda416c0 1 --2- 10.20.193.27:0/782565218 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa2d40b5710 0x7fa2d40d5bc0 secure :-1 s=READY pgs=216 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa2e80023e0 tx=0x7fa2e8005de0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:28.942 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.932+0000 7fa2e6ffd6c0 1 -- 10.20.193.27:0/782565218 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fa2f40a7550 con 0x7fa2f8122b20 2026-02-13T02:22:28.966 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.965+0000 7ff24a1846c0 1 -- 10.20.193.27:0/2227139836 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ff24410d380 con 0x7ff24410cfb0 2026-02-13T02:22:28.967 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.965+0000 7ff240ff96c0 1 -- 10.20.193.27:0/2227139836 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff2300ca650 con 0x7ff24410cfb0 2026-02-13T02:22:28.967 INFO:teuthology.orchestra.run.trial027.stdout:111669149713 2026-02-13T02:22:28.969 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.968+0000 7ff24a1846c0 1 -- 10.20.193.27:0/2227139836 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff2180b5690 msgr2=0x7ff2180d5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.969 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.968+0000 7ff24a1846c0 1 --2- 10.20.193.27:0/2227139836 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff2180b5690 0x7ff2180d5b40 secure :-1 s=READY pgs=212 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff2440fb570 tx=0x7ff234009260 comp rx=0 tx=0).stop 2026-02-13T02:22:28.970 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.968+0000 7ff24a1846c0 1 -- 10.20.193.27:0/2227139836 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff24410cfb0 msgr2=0x7ff2440f4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.970 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.968+0000 7ff24a1846c0 1 --2- 10.20.193.27:0/2227139836 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff24410cfb0 0x7ff2440f4be0 secure :-1 s=READY pgs=304 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff2300097f0 tx=0x7ff23002fad0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.970 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.968+0000 7ff24a1846c0 1 -- 10.20.193.27:0/2227139836 shutdown_connections 2026-02-13T02:22:28.970 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.968+0000 7ff24a1846c0 1 --2- 10.20.193.27:0/2227139836 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff2180b5690 0x7ff2180d5b40 unknown :-1 s=CLOSED pgs=212 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.970 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.968+0000 7ff24a1846c0 1 --2- 10.20.193.27:0/2227139836 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff2440f5210 0x7ff2440fa4e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.970 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.968+0000 7ff24a1846c0 1 --2- 10.20.193.27:0/2227139836 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff2441131c0 0x7ff244173a60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.970 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.968+0000 7ff24a1846c0 1 --2- 10.20.193.27:0/2227139836 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff24410cfb0 0x7ff2440f4be0 unknown :-1 s=CLOSED pgs=304 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.970 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.968+0000 7ff24a1846c0 1 -- 10.20.193.27:0/2227139836 >> 10.20.193.27:0/2227139836 conn(0x7ff2440ea8b0 msgr2=0x7ff2440605e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:28.970 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.968+0000 7ff24a1846c0 1 -- 10.20.193.27:0/2227139836 shutdown_connections 2026-02-13T02:22:28.970 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.968+0000 7ff24a1846c0 1 -- 10.20.193.27:0/2227139836 wait complete. 2026-02-13T02:22:28.971 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.970+0000 7fa5cb7fe6c0 1 -- 10.20.193.27:0/2594122028 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fa5fc179070 con 0x7fa5fc061050 2026-02-13T02:22:28.971 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.970+0000 7fa5f1ffb6c0 1 -- 10.20.193.27:0/2594122028 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fa5e40a9560 con 0x7fa5fc061050 2026-02-13T02:22:28.972 INFO:teuthology.orchestra.run.trial027.stdout:279172874246 2026-02-13T02:22:28.974 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.973+0000 7fa5cb7fe6c0 1 -- 10.20.193.27:0/2594122028 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa5d80be120 msgr2=0x7fa5d80de5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.974 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.973+0000 7fa5cb7fe6c0 1 --2- 10.20.193.27:0/2594122028 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa5d80be120 0x7fa5d80de5d0 secure :-1 s=READY pgs=211 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa5ec009970 tx=0x7fa5ec0023a0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.974 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.973+0000 7fa5cb7fe6c0 1 -- 10.20.193.27:0/2594122028 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa5fc061050 msgr2=0x7fa5fc161a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.975 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.973+0000 7fa5cb7fe6c0 1 --2- 10.20.193.27:0/2594122028 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa5fc061050 0x7fa5fc161a70 secure :-1 s=READY pgs=303 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa5e400b6c0 tx=0x7fa5e400bb90 comp rx=0 tx=0).stop 2026-02-13T02:22:28.975 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.973+0000 7fa5cb7fe6c0 1 -- 10.20.193.27:0/2594122028 shutdown_connections 2026-02-13T02:22:28.975 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.973+0000 7fa5cb7fe6c0 1 --2- 10.20.193.27:0/2594122028 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa5d80be120 0x7fa5d80de5d0 unknown :-1 s=CLOSED pgs=211 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.975 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.973+0000 7fa5cb7fe6c0 1 --2- 10.20.193.27:0/2594122028 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa5fc1556c0 0x7fa5fc166e30 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.975 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.973+0000 7fa5cb7fe6c0 1 --2- 10.20.193.27:0/2594122028 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa5fc061050 0x7fa5fc161a70 unknown :-1 s=CLOSED pgs=303 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.975 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.973+0000 7fa5cb7fe6c0 1 --2- 10.20.193.27:0/2594122028 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa5fc060740 0x7fa5fc161530 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.975 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.973+0000 7fa5cb7fe6c0 1 -- 10.20.193.27:0/2594122028 >> 10.20.193.27:0/2594122028 conn(0x7fa5fc0ea8b0 msgr2=0x7fa5fc10ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:28.975 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.973+0000 7fa5cb7fe6c0 1 -- 10.20.193.27:0/2594122028 shutdown_connections 2026-02-13T02:22:28.975 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.973+0000 7fa5cb7fe6c0 1 -- 10.20.193.27:0/2594122028 wait complete. 2026-02-13T02:22:28.987 INFO:tasks.cephadm.ceph_manager.ceph:need seq 30064771096 got 30064771095 for osd.0 2026-02-13T02:22:28.987 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574869 got 55834574868 for osd.2 2026-02-13T02:22:28.993 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.989+0000 7f9b6d92f6c0 1 -- 10.20.193.27:0/3921708558 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9b6810d9a0 msgr2=0x7f9b681129e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.993 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.989+0000 7f9b6d92f6c0 1 --2- 10.20.193.27:0/3921708558 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9b6810d9a0 0x7f9b681129e0 secure :-1 s=READY pgs=310 gs=1 cs=0 l=1 c_cookie=6892026260381241734 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9b500099a0 tx=0x7f9b5002ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:28.993 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.989+0000 7f9b6d92f6c0 1 -- 10.20.193.27:0/3921708558 shutdown_connections 2026-02-13T02:22:28.993 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.989+0000 7f9b6d92f6c0 1 --2- 10.20.193.27:0/3921708558 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9b681131c0 0x7f9b68173b60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.993 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.989+0000 7f9b6d92f6c0 1 --2- 10.20.193.27:0/3921708558 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9b6810d9a0 0x7f9b681129e0 unknown :-1 s=CLOSED pgs=310 gs=1 cs=0 l=1 c_cookie=6892026260381241734 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.993 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.989+0000 7f9b6d92f6c0 1 --2- 10.20.193.27:0/3921708558 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9b6810cfb0 0x7f9b6810d380 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.993 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.989+0000 7f9b6d92f6c0 1 -- 10.20.193.27:0/3921708558 >> 10.20.193.27:0/3921708558 conn(0x7f9b680ea8b0 msgr2=0x7f9b6810acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:28.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.989+0000 7f9b6d92f6c0 1 -- 10.20.193.27:0/3921708558 shutdown_connections 2026-02-13T02:22:28.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.989+0000 7f9b6d92f6c0 1 -- 10.20.193.27:0/3921708558 wait complete. 2026-02-13T02:22:28.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.989+0000 7f9b6d92f6c0 1 Processor -- start 2026-02-13T02:22:28.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.989+0000 7f9b6d92f6c0 1 -- start start 2026-02-13T02:22:28.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.990+0000 7f9b6d92f6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9b6810cfb0 0x7f9b680f91a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.990+0000 7f9b6d92f6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9b681131c0 0x7f9b68173a60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.990+0000 7f9b6d92f6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9b680f97d0 0x7f9b680feaa0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:28.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.990+0000 7f9b6d92f6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9b680f9ec0 con 0x7f9b680f97d0 2026-02-13T02:22:28.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.990+0000 7f9b6d92f6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9b680fefe0 con 0x7f9b6810cfb0 2026-02-13T02:22:28.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.990+0000 7f9b6d92f6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9b680ff1a0 con 0x7f9b681131c0 2026-02-13T02:22:28.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.990+0000 7f9b677fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9b680f97d0 0x7f9b680feaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.995 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.990+0000 7f9b66ffd6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9b6810cfb0 0x7f9b680f91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.995 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.990+0000 7f9b667fc6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9b681131c0 0x7f9b68173a60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:28.995 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.990+0000 7f9b66ffd6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9b6810cfb0 0x7f9b680f91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.27:49522/0 (socket says 10.20.193.27:49522) 2026-02-13T02:22:28.995 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.990+0000 7f9b66ffd6c0 1 -- 10.20.193.27:0/3158728324 learned_addr learned my addr 10.20.193.27:0/3158728324 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:28.995 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.990+0000 7f9b667fc6c0 1 -- 10.20.193.27:0/3158728324 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9b6810cfb0 msgr2=0x7f9b680f91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.995 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.990+0000 7f9b667fc6c0 1 --2- 10.20.193.27:0/3158728324 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9b6810cfb0 0x7f9b680f91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.995 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.990+0000 7f9b667fc6c0 1 -- 10.20.193.27:0/3158728324 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9b680f97d0 msgr2=0x7f9b680feaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.995 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.990+0000 7f9b667fc6c0 1 --2- 10.20.193.27:0/3158728324 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9b680f97d0 0x7f9b680feaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.995 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.990+0000 7f9b667fc6c0 1 -- 10.20.193.27:0/3158728324 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b50009590 con 0x7f9b681131c0 2026-02-13T02:22:28.995 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.990+0000 7f9b677fe6c0 1 --2- 10.20.193.27:0/3158728324 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9b680f97d0 0x7f9b680feaa0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:22:28.995 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.990+0000 7f9b667fc6c0 1 --2- 10.20.193.27:0/3158728324 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9b681131c0 0x7f9b68173a60 secure :-1 s=READY pgs=58 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9b500099a0 tx=0x7f9b5002fc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:28.995 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.990+0000 7f9b6c9296c0 1 -- 10.20.193.27:0/3158728324 <== mon.1 v2:10.20.193.107:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9b50002ae0 con 0x7f9b681131c0 2026-02-13T02:22:28.995 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.990+0000 7f9b6c9296c0 1 -- 10.20.193.27:0/3158728324 <== mon.1 v2:10.20.193.107:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9b50005e70 con 0x7f9b681131c0 2026-02-13T02:22:28.996 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.990+0000 7f9b6d92f6c0 1 -- 10.20.193.27:0/3158728324 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b680ff420 con 0x7f9b681131c0 2026-02-13T02:22:28.996 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.990+0000 7f9b6c9296c0 1 -- 10.20.193.27:0/3158728324 <== mon.1 v2:10.20.193.107:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9b50036070 con 0x7f9b681131c0 2026-02-13T02:22:28.996 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.990+0000 7f9b6d92f6c0 1 -- 10.20.193.27:0/3158728324 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b680ff870 con 0x7f9b681131c0 2026-02-13T02:22:28.997 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.990+0000 7fa1502e76c0 1 -- 10.20.193.27:0/1871832709 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa148115440 msgr2=0x7fa148175de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:28.997 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.990+0000 7fa1502e76c0 1 --2- 10.20.193.27:0/1871832709 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa148115440 0x7fa148175de0 secure :-1 s=READY pgs=311 gs=3 cs=0 l=1 c_cookie=5578073508186737474 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa14400aa70 tx=0x7fa14402fde0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.997 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.990+0000 7fa1502e76c0 1 -- 10.20.193.27:0/1871832709 shutdown_connections 2026-02-13T02:22:28.997 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.990+0000 7fa1502e76c0 1 --2- 10.20.193.27:0/1871832709 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa148115440 0x7fa148175de0 unknown :-1 s=CLOSED pgs=311 gs=3 cs=0 l=1 c_cookie=5578073508186737474 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.997 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.990+0000 7fa1502e76c0 1 --2- 10.20.193.27:0/1871832709 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa14810fc20 0x7fa148114c60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.997 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.990+0000 7fa1502e76c0 1 --2- 10.20.193.27:0/1871832709 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa14810f230 0x7fa14810f600 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:28.997 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.990+0000 7fa1502e76c0 1 -- 10.20.193.27:0/1871832709 >> 10.20.193.27:0/1871832709 conn(0x7fa1480ecb30 msgr2=0x7fa14810cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:28.997 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.991+0000 7fa1502e76c0 1 -- 10.20.193.27:0/1871832709 shutdown_connections 2026-02-13T02:22:28.997 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.991+0000 7fa1502e76c0 1 -- 10.20.193.27:0/1871832709 wait complete. 2026-02-13T02:22:28.998 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.991+0000 7fa1502e76c0 1 Processor -- start 2026-02-13T02:22:28.998 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.991+0000 7f9b6d92f6c0 1 -- 10.20.193.27:0/3158728324 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b6810e280 con 0x7f9b681131c0 2026-02-13T02:22:29.000 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.991+0000 7fa1502e76c0 1 -- start start 2026-02-13T02:22:29.000 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.991+0000 7fa1502e76c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa14810f230 0x7fa14811d280 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:29.000 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.991+0000 7fa1502e76c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa148115440 0x7fa14811d7c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:29.000 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.991+0000 7fa1502e76c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa148122b80 0x7fa148123020 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:29.000 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.991+0000 7fa1502e76c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa148123590 con 0x7fa14810f230 2026-02-13T02:22:29.000 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.991+0000 7fa1502e76c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa148123700 con 0x7fa148115440 2026-02-13T02:22:29.000 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.991+0000 7fa1502e76c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa1481238c0 con 0x7fa148122b80 2026-02-13T02:22:29.001 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.992+0000 7fa14e0586c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa14810f230 0x7fa14811d280 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:29.001 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.992+0000 7fa14e0586c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa14810f230 0x7fa14811d280 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:53778/0 (socket says 10.20.193.27:53778) 2026-02-13T02:22:29.001 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.992+0000 7fa14e0586c0 1 -- 10.20.193.27:0/3533716426 learned_addr learned my addr 10.20.193.27:0/3533716426 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:29.001 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.992+0000 7fa14d8576c0 1 --2- 10.20.193.27:0/3533716426 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa148115440 0x7fa14811d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:29.001 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.992+0000 7fa14e8596c0 1 --2- 10.20.193.27:0/3533716426 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa148122b80 0x7fa148123020 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:29.001 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.992+0000 7fa14e0586c0 1 -- 10.20.193.27:0/3533716426 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa148122b80 msgr2=0x7fa148123020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:29.001 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.992+0000 7fa14e0586c0 1 --2- 10.20.193.27:0/3533716426 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa148122b80 0x7fa148123020 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.001 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.992+0000 7fa14e0586c0 1 -- 10.20.193.27:0/3533716426 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa148115440 msgr2=0x7fa14811d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:29.001 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.992+0000 7fa14e0586c0 1 --2- 10.20.193.27:0/3533716426 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa148115440 0x7fa14811d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.001 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.992+0000 7fa14e0586c0 1 -- 10.20.193.27:0/3533716426 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14400a660 con 0x7fa14810f230 2026-02-13T02:22:29.001 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.992+0000 7fa14e0586c0 1 --2- 10.20.193.27:0/3533716426 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa14810f230 0x7fa14811d280 secure :-1 s=READY pgs=312 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa138009490 tx=0x7fa13800bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:29.001 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.992+0000 7fa12f7fe6c0 1 -- 10.20.193.27:0/3533716426 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa138004a80 con 0x7fa14810f230 2026-02-13T02:22:29.002 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.992+0000 7fa12f7fe6c0 1 -- 10.20.193.27:0/3533716426 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa138009030 con 0x7fa14810f230 2026-02-13T02:22:29.002 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.992+0000 7fa1502e76c0 1 -- 10.20.193.27:0/3533716426 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa148163d70 con 0x7fa14810f230 2026-02-13T02:22:29.002 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.992+0000 7fa12f7fe6c0 1 -- 10.20.193.27:0/3533716426 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa13800ce30 con 0x7fa14810f230 2026-02-13T02:22:29.002 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.992+0000 7fa1502e76c0 1 -- 10.20.193.27:0/3533716426 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa148164250 con 0x7fa14810f230 2026-02-13T02:22:29.002 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:29.003 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.992+0000 7f9b6c9296c0 1 -- 10.20.193.27:0/3158728324 <== mon.1 v2:10.20.193.107:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f9b500071c0 con 0x7f9b681131c0 2026-02-13T02:22:29.003 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.993+0000 7f9b6c9296c0 1 --2- 10.20.193.27:0/3158728324 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9b3c0b5690 0x7f9b3c0d5b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:29.003 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.993+0000 7f9b66ffd6c0 1 --2- 10.20.193.27:0/3158728324 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9b3c0b5690 0x7f9b3c0d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:29.003 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.993+0000 7fa1502e76c0 1 -- 10.20.193.27:0/3533716426 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa148110500 con 0x7fa14810f230 2026-02-13T02:22:29.004 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.993+0000 7f9b6c9296c0 1 -- 10.20.193.27:0/3158728324 <== mon.1 v2:10.20.193.107:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f9b50107b50 con 0x7f9b681131c0 2026-02-13T02:22:29.004 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.993+0000 7f9b66ffd6c0 1 --2- 10.20.193.27:0/3158728324 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9b3c0b5690 0x7f9b3c0d5b40 secure :-1 s=READY pgs=217 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9b5c0095e0 tx=0x7f9b5c009260 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:29.004 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.994+0000 7fa12f7fe6c0 1 -- 10.20.193.27:0/3533716426 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7fa138004060 con 0x7fa14810f230 2026-02-13T02:22:29.005 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.994+0000 7f9b6c9296c0 1 -- 10.20.193.27:0/3158728324 <== mon.1 v2:10.20.193.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f9b500ca5b0 con 0x7f9b681131c0 2026-02-13T02:22:29.005 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.995+0000 7fa12f7fe6c0 1 --2- 10.20.193.27:0/3533716426 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa1200b5710 0x7fa1200d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:29.005 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.995+0000 7fa14d8576c0 1 --2- 10.20.193.27:0/3533716426 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa1200b5710 0x7fa1200d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:29.005 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.995+0000 7fa12f7fe6c0 1 -- 10.20.193.27:0/3533716426 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7fa1380e3af0 con 0x7fa14810f230 2026-02-13T02:22:29.005 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.996+0000 7fa14d8576c0 1 --2- 10.20.193.27:0/3533716426 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa1200b5710 0x7fa1200d5bc0 secure :-1 s=READY pgs=218 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa148123c10 tx=0x7fa13c009390 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:29.006 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.997+0000 7f27687fb6c0 1 -- 10.20.193.27:0/358238879 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f276010cfb0 msgr2=0x7f276010d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:29.006 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.997+0000 7f27687fb6c0 1 --2- 10.20.193.27:0/358238879 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f276010cfb0 0x7f276010d380 secure :-1 s=READY pgs=313 gs=1 cs=0 l=1 c_cookie=6859616870742727586 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f27500098d0 tx=0x7f275002ec40 comp rx=0 tx=0).stop 2026-02-13T02:22:29.006 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.997+0000 7f27687fb6c0 1 -- 10.20.193.27:0/358238879 shutdown_connections 2026-02-13T02:22:29.006 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.997+0000 7f27687fb6c0 1 --2- 10.20.193.27:0/358238879 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f27601131c0 0x7f2760173b60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.006 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.997+0000 7f27687fb6c0 1 --2- 10.20.193.27:0/358238879 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f276010d9a0 0x7f27601129e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.006 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.997+0000 7f27687fb6c0 1 --2- 10.20.193.27:0/358238879 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f276010cfb0 0x7f276010d380 unknown :-1 s=CLOSED pgs=313 gs=1 cs=0 l=1 c_cookie=6859616870742727586 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.006 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.998+0000 7f27687fb6c0 1 -- 10.20.193.27:0/358238879 >> 10.20.193.27:0/358238879 conn(0x7f27600ea8b0 msgr2=0x7f276010acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:29.006 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.998+0000 7f27687fb6c0 1 -- 10.20.193.27:0/358238879 shutdown_connections 2026-02-13T02:22:29.007 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.998+0000 7f27687fb6c0 1 -- 10.20.193.27:0/358238879 wait complete. 2026-02-13T02:22:29.007 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.998+0000 7fa12f7fe6c0 1 -- 10.20.193.27:0/3533716426 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fa1380a6510 con 0x7fa14810f230 2026-02-13T02:22:29.010 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.998+0000 7f27687fb6c0 1 Processor -- start 2026-02-13T02:22:29.010 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.998+0000 7f27687fb6c0 1 -- start start 2026-02-13T02:22:29.010 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.999+0000 7f27687fb6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f276010cfb0 0x7f276011b020 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:29.010 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.999+0000 7f27687fb6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f27601131c0 0x7f276011b560 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:29.010 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.999+0000 7f27687fb6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2760120920 0x7f2760120dc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:29.010 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.999+0000 7f27687fb6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2760121330 con 0x7f276010cfb0 2026-02-13T02:22:29.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.999+0000 7f27687fb6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f27601214a0 con 0x7f27601131c0 2026-02-13T02:22:29.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.999+0000 7f276656c6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f276010cfb0 0x7f276011b020 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:29.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.999+0000 7f27687fb6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2760121660 con 0x7f2760120920 2026-02-13T02:22:29.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.999+0000 7f276656c6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f276010cfb0 0x7f276011b020 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:53810/0 (socket says 10.20.193.27:53810) 2026-02-13T02:22:29.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.999+0000 7f276656c6c0 1 -- 10.20.193.27:0/2744320861 learned_addr learned my addr 10.20.193.27:0/2744320861 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:29.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.999+0000 7f2765d6b6c0 1 --2- 10.20.193.27:0/2744320861 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f27601131c0 0x7f276011b560 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:29.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.999+0000 7f2766d6d6c0 1 --2- 10.20.193.27:0/2744320861 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2760120920 0x7f2760120dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:29.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.999+0000 7f2765d6b6c0 1 -- 10.20.193.27:0/2744320861 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2760120920 msgr2=0x7f2760120dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:29.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.999+0000 7f2765d6b6c0 1 --2- 10.20.193.27:0/2744320861 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2760120920 0x7f2760120dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.999+0000 7f2765d6b6c0 1 -- 10.20.193.27:0/2744320861 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f276010cfb0 msgr2=0x7f276011b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:29.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.999+0000 7f2765d6b6c0 1 --2- 10.20.193.27:0/2744320861 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f276010cfb0 0x7f276011b020 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.999+0000 7f2765d6b6c0 1 -- 10.20.193.27:0/2744320861 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27500094c0 con 0x7f27601131c0 2026-02-13T02:22:29.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.999+0000 7f276656c6c0 1 --2- 10.20.193.27:0/2744320861 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f276010cfb0 0x7f276011b020 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-13T02:22:29.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.999+0000 7f2765d6b6c0 1 --2- 10.20.193.27:0/2744320861 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f27601131c0 0x7f276011b560 secure :-1 s=READY pgs=58 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2754004750 tx=0x7f275400bb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:29.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.999+0000 7f27477fe6c0 1 -- 10.20.193.27:0/2744320861 <== mon.2 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2754004a80 con 0x7f27601131c0 2026-02-13T02:22:29.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.999+0000 7f27477fe6c0 1 -- 10.20.193.27:0/2744320861 <== mon.2 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f2754009030 con 0x7f27601131c0 2026-02-13T02:22:29.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.999+0000 7f27687fb6c0 1 -- 10.20.193.27:0/2744320861 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2760161b10 con 0x7f27601131c0 2026-02-13T02:22:29.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.999+0000 7f27477fe6c0 1 -- 10.20.193.27:0/2744320861 <== mon.2 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f275400cce0 con 0x7f27601131c0 2026-02-13T02:22:29.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:28.999+0000 7f27687fb6c0 1 -- 10.20.193.27:0/2744320861 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2760161fa0 con 0x7f27601131c0 2026-02-13T02:22:29.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.000+0000 7f27687fb6c0 1 -- 10.20.193.27:0/2744320861 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f276010e280 con 0x7f27601131c0 2026-02-13T02:22:29.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.001+0000 7f27477fe6c0 1 -- 10.20.193.27:0/2744320861 <== mon.2 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f2754004060 con 0x7f27601131c0 2026-02-13T02:22:29.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.001+0000 7f27477fe6c0 1 --2- 10.20.193.27:0/2744320861 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f273c0b5690 0x7f273c0d5b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:29.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.002+0000 7f27477fe6c0 1 -- 10.20.193.27:0/2744320861 <== mon.2 v2:10.20.193.28:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f27540e3b50 con 0x7f27601131c0 2026-02-13T02:22:29.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.002+0000 7f276656c6c0 1 --2- 10.20.193.27:0/2744320861 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f273c0b5690 0x7f273c0d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:29.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.002+0000 7f27477fe6c0 1 -- 10.20.193.27:0/2744320861 <== mon.2 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f27540a6570 con 0x7f27601131c0 2026-02-13T02:22:29.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.002+0000 7f276656c6c0 1 --2- 10.20.193.27:0/2744320861 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f273c0b5690 0x7f273c0d5b40 secure :-1 s=READY pgs=219 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f27500097f0 tx=0x7f2750005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:29.023 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.022+0000 7f3dd88276c0 1 -- 10.20.193.27:0/742511478 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f3dd010f630 con 0x7f3dd00f4810 2026-02-13T02:22:29.024 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.022+0000 7f3dbb7fe6c0 1 -- 10.20.193.27:0/742511478 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3dc40aa870 con 0x7f3dd00f4810 2026-02-13T02:22:29.024 INFO:teuthology.orchestra.run.trial027.stdout:81604378643 2026-02-13T02:22:29.027 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149714 got 111669149713 for osd.4 2026-02-13T02:22:29.027 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.025+0000 7f3dd88276c0 1 -- 10.20.193.27:0/742511478 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3dac0b5590 msgr2=0x7f3dac0d5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:29.027 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.025+0000 7f3dd88276c0 1 --2- 10.20.193.27:0/742511478 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3dac0b5590 0x7f3dac0d5a40 secure :-1 s=READY pgs=214 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3dc8004750 tx=0x7f3dc8009260 comp rx=0 tx=0).stop 2026-02-13T02:22:29.027 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.025+0000 7f3dd88276c0 1 -- 10.20.193.27:0/742511478 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3dd00f4810 msgr2=0x7f3dd00f9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:29.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.025+0000 7f3dd88276c0 1 --2- 10.20.193.27:0/742511478 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3dd00f4810 0x7f3dd00f9ae0 secure :-1 s=READY pgs=308 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3dc4006ea0 tx=0x7f3dc400ce50 comp rx=0 tx=0).stop 2026-02-13T02:22:29.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.025+0000 7f3dd88276c0 1 -- 10.20.193.27:0/742511478 shutdown_connections 2026-02-13T02:22:29.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.025+0000 7f3dd88276c0 1 --2- 10.20.193.27:0/742511478 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3dac0b5590 0x7f3dac0d5a40 unknown :-1 s=CLOSED pgs=214 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.025+0000 7f3dd88276c0 1 --2- 10.20.193.27:0/742511478 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3dd00f4810 0x7f3dd00f9ae0 unknown :-1 s=CLOSED pgs=308 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.025+0000 7f3dd88276c0 1 --2- 10.20.193.27:0/742511478 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3dd0115410 0x7f3dd0175cb0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.025+0000 7f3dd88276c0 1 --2- 10.20.193.27:0/742511478 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3dd010f200 0x7f3dd00f41e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.025+0000 7f3dd88276c0 1 -- 10.20.193.27:0/742511478 >> 10.20.193.27:0/742511478 conn(0x7f3dd00ecb00 msgr2=0x7f3dd0113230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:29.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.025+0000 7f3dd88276c0 1 -- 10.20.193.27:0/742511478 shutdown_connections 2026-02-13T02:22:29.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.025+0000 7f3dd88276c0 1 -- 10.20.193.27:0/742511478 wait complete. 2026-02-13T02:22:29.029 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.025+0000 7f66200406c0 1 -- 10.20.193.27:0/1952203966 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f661811e3b0 con 0x7f661810f230 2026-02-13T02:22:29.029 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.026+0000 7f6606ffd6c0 1 -- 10.20.193.27:0/1952203966 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f660800ce60 con 0x7f661810f230 2026-02-13T02:22:29.029 INFO:teuthology.orchestra.run.trial027.stdout:201863462923 2026-02-13T02:22:29.029 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.028+0000 7fa3157fa6c0 1 -- 10.20.193.27:0/1981214267 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fa2f8005890 con 0x7fa33010f210 2026-02-13T02:22:29.030 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.029+0000 7fa3177fe6c0 1 -- 10.20.193.27:0/1981214267 <== mon.2 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa3240cd750 con 0x7fa33010f210 2026-02-13T02:22:29.030 INFO:teuthology.orchestra.run.trial027.stdout:171798691852 2026-02-13T02:22:29.031 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.029+0000 7f66200406c0 1 -- 10.20.193.27:0/1952203966 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f65f40b5690 msgr2=0x7f65f40d5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:29.031 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.029+0000 7f66200406c0 1 --2- 10.20.193.27:0/1952203966 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f65f40b5690 0x7f65f40d5b40 secure :-1 s=READY pgs=213 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f660c005980 tx=0x7f660c005910 comp rx=0 tx=0).stop 2026-02-13T02:22:29.031 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.029+0000 7f66200406c0 1 -- 10.20.193.27:0/1952203966 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f661810f230 msgr2=0x7f661811d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:29.031 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.029+0000 7f66200406c0 1 --2- 10.20.193.27:0/1952203966 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f661810f230 0x7f661811d200 secure :-1 s=READY pgs=307 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f660800b710 tx=0x7f660800bbe0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.031 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.029+0000 7f66200406c0 1 -- 10.20.193.27:0/1952203966 shutdown_connections 2026-02-13T02:22:29.031 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.029+0000 7f66200406c0 1 --2- 10.20.193.27:0/1952203966 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f65f40b5690 0x7f65f40d5b40 unknown :-1 s=CLOSED pgs=213 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.032 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.029+0000 7f66200406c0 1 --2- 10.20.193.27:0/1952203966 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6618122b00 0x7f6618122fa0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.032 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.029+0000 7f66200406c0 1 --2- 10.20.193.27:0/1952203966 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f6618115440 0x7f661811d740 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.032 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.029+0000 7f66200406c0 1 --2- 10.20.193.27:0/1952203966 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f661810f230 0x7f661811d200 unknown :-1 s=CLOSED pgs=307 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.032 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.029+0000 7f66200406c0 1 -- 10.20.193.27:0/1952203966 >> 10.20.193.27:0/1952203966 conn(0x7f66180ecb30 msgr2=0x7f661810cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:29.032 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.029+0000 7f66200406c0 1 -- 10.20.193.27:0/1952203966 shutdown_connections 2026-02-13T02:22:29.032 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.029+0000 7f66200406c0 1 -- 10.20.193.27:0/1952203966 wait complete. 2026-02-13T02:22:29.033 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.031+0000 7fa3157fa6c0 1 -- 10.20.193.27:0/1981214267 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa3080b5710 msgr2=0x7fa3080d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:29.033 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.031+0000 7fa3157fa6c0 1 --2- 10.20.193.27:0/1981214267 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa3080b5710 0x7fa3080d5bc0 secure :-1 s=READY pgs=215 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa318005980 tx=0x7fa318005910 comp rx=0 tx=0).stop 2026-02-13T02:22:29.033 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.031+0000 7fa3157fa6c0 1 -- 10.20.193.27:0/1981214267 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa33010f210 msgr2=0x7fa33011d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:29.033 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.031+0000 7fa3157fa6c0 1 --2- 10.20.193.27:0/1981214267 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa33010f210 0x7fa33011d260 secure :-1 s=READY pgs=56 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa32402f230 tx=0x7fa32402fd40 comp rx=0 tx=0).stop 2026-02-13T02:22:29.033 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.031+0000 7fa3157fa6c0 1 -- 10.20.193.27:0/1981214267 shutdown_connections 2026-02-13T02:22:29.034 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.031+0000 7fa3157fa6c0 1 --2- 10.20.193.27:0/1981214267 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa3080b5710 0x7fa3080d5bc0 unknown :-1 s=CLOSED pgs=215 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.034 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.031+0000 7fa3157fa6c0 1 --2- 10.20.193.27:0/1981214267 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa330122b60 0x7fa330123000 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.034 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.031+0000 7fa3157fa6c0 1 --2- 10.20.193.27:0/1981214267 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa330115420 0x7fa33011d7a0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.034 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.031+0000 7fa3157fa6c0 1 --2- 10.20.193.27:0/1981214267 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa33010f210 0x7fa33011d260 unknown :-1 s=CLOSED pgs=56 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.034 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.031+0000 7fa3157fa6c0 1 -- 10.20.193.27:0/1981214267 >> 10.20.193.27:0/1981214267 conn(0x7fa3300ecb30 msgr2=0x7fa33010cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:29.034 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.032+0000 7fa3157fa6c0 1 -- 10.20.193.27:0/1981214267 shutdown_connections 2026-02-13T02:22:29.034 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.032+0000 7fa3157fa6c0 1 -- 10.20.193.27:0/1981214267 wait complete. 2026-02-13T02:22:29.037 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874247 got 279172874246 for osd.11 2026-02-13T02:22:29.044 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.043+0000 7fa2e4ff96c0 1 -- 10.20.193.27:0/782565218 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fa2c4005890 con 0x7fa2f8122b20 2026-02-13T02:22:29.044 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.043+0000 7fa2e6ffd6c0 1 -- 10.20.193.27:0/782565218 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa2f40aa700 con 0x7fa2f8122b20 2026-02-13T02:22:29.045 INFO:teuthology.orchestra.run.trial027.stdout:231928233994 2026-02-13T02:22:29.047 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.045+0000 7fa2e4ff96c0 1 -- 10.20.193.27:0/782565218 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa2d40b5710 msgr2=0x7fa2d40d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:29.047 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.045+0000 7fa2e4ff96c0 1 --2- 10.20.193.27:0/782565218 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa2d40b5710 0x7fa2d40d5bc0 secure :-1 s=READY pgs=216 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa2e80023e0 tx=0x7fa2e8005de0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.047 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.045+0000 7fa2e4ff96c0 1 -- 10.20.193.27:0/782565218 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa2f8122b20 msgr2=0x7fa2f8122fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:29.047 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.045+0000 7fa2e4ff96c0 1 --2- 10.20.193.27:0/782565218 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa2f8122b20 0x7fa2f8122fc0 secure :-1 s=READY pgs=309 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa2f4006ea0 tx=0x7fa2f400ce50 comp rx=0 tx=0).stop 2026-02-13T02:22:29.047 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.045+0000 7fa2e4ff96c0 1 -- 10.20.193.27:0/782565218 shutdown_connections 2026-02-13T02:22:29.048 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.045+0000 7fa2e4ff96c0 1 --2- 10.20.193.27:0/782565218 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa2d40b5710 0x7fa2d40d5bc0 unknown :-1 s=CLOSED pgs=216 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.048 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.045+0000 7fa2e4ff96c0 1 --2- 10.20.193.27:0/782565218 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa2f8122b20 0x7fa2f8122fc0 unknown :-1 s=CLOSED pgs=309 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.048 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.045+0000 7fa2e4ff96c0 1 --2- 10.20.193.27:0/782565218 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa2f8115440 0x7fa2f811d760 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.048 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.045+0000 7fa2e4ff96c0 1 --2- 10.20.193.27:0/782565218 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa2f810f230 0x7fa2f811d220 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.048 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.046+0000 7fa2e4ff96c0 1 -- 10.20.193.27:0/782565218 >> 10.20.193.27:0/782565218 conn(0x7fa2f80ecb30 msgr2=0x7fa2f810cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:29.048 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.046+0000 7fa2e4ff96c0 1 -- 10.20.193.27:0/782565218 shutdown_connections 2026-02-13T02:22:29.048 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.046+0000 7fa2e4ff96c0 1 -- 10.20.193.27:0/782565218 wait complete. 2026-02-13T02:22:29.074 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:29 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1542532650' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2026-02-13T02:22:29.075 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:29 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2104342111' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2026-02-13T02:22:29.075 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:29 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2227139836' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2026-02-13T02:22:29.075 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:29 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2594122028' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2026-02-13T02:22:29.075 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:29 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/742511478' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2026-02-13T02:22:29.075 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:29 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1952203966' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2026-02-13T02:22:29.075 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:29 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1981214267' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2026-02-13T02:22:29.075 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:29 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/782565218' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2026-02-13T02:22:29.080 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378644 got 81604378643 for osd.3 2026-02-13T02:22:29.082 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462924 got 201863462923 for osd.8 2026-02-13T02:22:29.086 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691853 got 171798691852 for osd.7 2026-02-13T02:22:29.099 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928233995 got 231928233994 for osd.9 2026-02-13T02:22:29.102 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.101+0000 7f9b6d92f6c0 1 -- 10.20.193.27:0/3158728324 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f9b680fa2f0 con 0x7f9b681131c0 2026-02-13T02:22:29.103 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.101+0000 7f9b6c9296c0 1 -- 10.20.193.27:0/3158728324 <== mon.1 v2:10.20.193.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9b50005c00 con 0x7f9b681131c0 2026-02-13T02:22:29.103 INFO:teuthology.orchestra.run.trial027.stdout:42949672983 2026-02-13T02:22:29.105 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.104+0000 7f9b6d92f6c0 1 -- 10.20.193.27:0/3158728324 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9b3c0b5690 msgr2=0x7f9b3c0d5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:29.106 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.104+0000 7f9b6d92f6c0 1 --2- 10.20.193.27:0/3158728324 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9b3c0b5690 0x7f9b3c0d5b40 secure :-1 s=READY pgs=217 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9b5c0095e0 tx=0x7f9b5c009260 comp rx=0 tx=0).stop 2026-02-13T02:22:29.106 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.104+0000 7f9b6d92f6c0 1 -- 10.20.193.27:0/3158728324 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9b681131c0 msgr2=0x7f9b68173a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:29.106 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.104+0000 7f9b6d92f6c0 1 --2- 10.20.193.27:0/3158728324 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9b681131c0 0x7f9b68173a60 secure :-1 s=READY pgs=58 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9b500099a0 tx=0x7f9b5002fc00 comp rx=0 tx=0).stop 2026-02-13T02:22:29.106 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.104+0000 7f9b6d92f6c0 1 -- 10.20.193.27:0/3158728324 shutdown_connections 2026-02-13T02:22:29.106 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.104+0000 7f9b6d92f6c0 1 --2- 10.20.193.27:0/3158728324 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9b3c0b5690 0x7f9b3c0d5b40 unknown :-1 s=CLOSED pgs=217 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.106 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.104+0000 7f9b6d92f6c0 1 --2- 10.20.193.27:0/3158728324 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9b680f97d0 0x7f9b680feaa0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.106 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.104+0000 7f9b6d92f6c0 1 --2- 10.20.193.27:0/3158728324 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9b681131c0 0x7f9b68173a60 unknown :-1 s=CLOSED pgs=58 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.106 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.104+0000 7f9b6d92f6c0 1 --2- 10.20.193.27:0/3158728324 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9b6810cfb0 0x7f9b680f91a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.106 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.104+0000 7f9b6d92f6c0 1 -- 10.20.193.27:0/3158728324 >> 10.20.193.27:0/3158728324 conn(0x7f9b680ea8b0 msgr2=0x7f9b6810b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:29.106 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.104+0000 7f9b6d92f6c0 1 -- 10.20.193.27:0/3158728324 shutdown_connections 2026-02-13T02:22:29.106 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.104+0000 7f9b6d92f6c0 1 -- 10.20.193.27:0/3158728324 wait complete. 2026-02-13T02:22:29.107 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.105+0000 7f27687fb6c0 1 -- 10.20.193.27:0/2744320861 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f276010d9a0 con 0x7f27601131c0 2026-02-13T02:22:29.107 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.105+0000 7f27477fe6c0 1 -- 10.20.193.27:0/2744320861 <== mon.2 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f27540a9720 con 0x7f27601131c0 2026-02-13T02:22:29.107 INFO:teuthology.orchestra.run.trial027.stdout:128849018897 2026-02-13T02:22:29.110 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.108+0000 7f27687fb6c0 1 -- 10.20.193.27:0/2744320861 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f273c0b5690 msgr2=0x7f273c0d5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:29.110 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.108+0000 7f27687fb6c0 1 --2- 10.20.193.27:0/2744320861 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f273c0b5690 0x7f273c0d5b40 secure :-1 s=READY pgs=219 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f27500097f0 tx=0x7f2750005910 comp rx=0 tx=0).stop 2026-02-13T02:22:29.110 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.108+0000 7f27687fb6c0 1 -- 10.20.193.27:0/2744320861 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f27601131c0 msgr2=0x7f276011b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:29.110 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.108+0000 7f27687fb6c0 1 --2- 10.20.193.27:0/2744320861 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f27601131c0 0x7f276011b560 secure :-1 s=READY pgs=58 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2754004750 tx=0x7f275400bb80 comp rx=0 tx=0).stop 2026-02-13T02:22:29.110 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.108+0000 7f27687fb6c0 1 -- 10.20.193.27:0/2744320861 shutdown_connections 2026-02-13T02:22:29.110 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.108+0000 7f27687fb6c0 1 --2- 10.20.193.27:0/2744320861 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f273c0b5690 0x7f273c0d5b40 unknown :-1 s=CLOSED pgs=219 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.110 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.108+0000 7f27687fb6c0 1 --2- 10.20.193.27:0/2744320861 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2760120920 0x7f2760120dc0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.110 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.108+0000 7f27687fb6c0 1 --2- 10.20.193.27:0/2744320861 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f27601131c0 0x7f276011b560 unknown :-1 s=CLOSED pgs=58 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.110 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.108+0000 7f27687fb6c0 1 --2- 10.20.193.27:0/2744320861 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f276010cfb0 0x7f276011b020 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.111 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.108+0000 7f27687fb6c0 1 -- 10.20.193.27:0/2744320861 >> 10.20.193.27:0/2744320861 conn(0x7f27600ea8b0 msgr2=0x7f276010ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:29.111 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.108+0000 7f27687fb6c0 1 -- 10.20.193.27:0/2744320861 shutdown_connections 2026-02-13T02:22:29.111 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.108+0000 7f27687fb6c0 1 -- 10.20.193.27:0/2744320861 wait complete. 2026-02-13T02:22:29.112 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.111+0000 7fa1502e76c0 1 -- 10.20.193.27:0/3533716426 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fa14811e430 con 0x7fa14810f230 2026-02-13T02:22:29.113 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.111+0000 7fa12f7fe6c0 1 -- 10.20.193.27:0/3533716426 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa1380a96c0 con 0x7fa14810f230 2026-02-13T02:22:29.113 INFO:teuthology.orchestra.run.trial027.stdout:158913789968 2026-02-13T02:22:29.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.113+0000 7fa1502e76c0 1 -- 10.20.193.27:0/3533716426 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa1200b5710 msgr2=0x7fa1200d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:29.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.113+0000 7fa1502e76c0 1 --2- 10.20.193.27:0/3533716426 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa1200b5710 0x7fa1200d5bc0 secure :-1 s=READY pgs=218 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa148123c10 tx=0x7fa13c009390 comp rx=0 tx=0).stop 2026-02-13T02:22:29.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.113+0000 7fa1502e76c0 1 -- 10.20.193.27:0/3533716426 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa14810f230 msgr2=0x7fa14811d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:29.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.113+0000 7fa1502e76c0 1 --2- 10.20.193.27:0/3533716426 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa14810f230 0x7fa14811d280 secure :-1 s=READY pgs=312 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa138009490 tx=0x7fa13800bca0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.113+0000 7fa1502e76c0 1 -- 10.20.193.27:0/3533716426 shutdown_connections 2026-02-13T02:22:29.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.113+0000 7fa1502e76c0 1 --2- 10.20.193.27:0/3533716426 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa1200b5710 0x7fa1200d5bc0 unknown :-1 s=CLOSED pgs=218 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.113+0000 7fa1502e76c0 1 --2- 10.20.193.27:0/3533716426 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa148122b80 0x7fa148123020 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.113+0000 7fa1502e76c0 1 --2- 10.20.193.27:0/3533716426 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa148115440 0x7fa14811d7c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.113+0000 7fa1502e76c0 1 --2- 10.20.193.27:0/3533716426 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa14810f230 0x7fa14811d280 unknown :-1 s=CLOSED pgs=312 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.116 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.113+0000 7fa1502e76c0 1 -- 10.20.193.27:0/3533716426 >> 10.20.193.27:0/3533716426 conn(0x7fa1480ecb30 msgr2=0x7fa14810cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:29.116 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.113+0000 7fa1502e76c0 1 -- 10.20.193.27:0/3533716426 shutdown_connections 2026-02-13T02:22:29.116 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.113+0000 7fa1502e76c0 1 -- 10.20.193.27:0/3533716426 wait complete. 2026-02-13T02:22:29.143 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.141+0000 7f8ec4ad06c0 1 -- 10.20.193.27:0/669268653 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8ec0115410 msgr2=0x7f8ec0175db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:29.143 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.141+0000 7f8ec4ad06c0 1 --2- 10.20.193.27:0/669268653 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8ec0115410 0x7f8ec0175db0 secure :-1 s=READY pgs=314 gs=1 cs=0 l=1 c_cookie=18296802216201455514 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8eb000aa70 tx=0x7f8eb002fde0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.143 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.141+0000 7f8ec4ad06c0 1 -- 10.20.193.27:0/669268653 shutdown_connections 2026-02-13T02:22:29.143 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.141+0000 7f8ec4ad06c0 1 --2- 10.20.193.27:0/669268653 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8ec0115410 0x7f8ec0175db0 unknown :-1 s=CLOSED pgs=314 gs=1 cs=0 l=1 c_cookie=18296802216201455514 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.143 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.141+0000 7f8ec4ad06c0 1 --2- 10.20.193.27:0/669268653 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8ec010fbf0 0x7f8ec0114c30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.143 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.141+0000 7f8ec4ad06c0 1 --2- 10.20.193.27:0/669268653 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8ec010f200 0x7f8ec010f5d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.143 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.141+0000 7f8ec4ad06c0 1 -- 10.20.193.27:0/669268653 >> 10.20.193.27:0/669268653 conn(0x7f8ec00ecb00 msgr2=0x7f8ec010cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:29.144 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.141+0000 7f8ec4ad06c0 1 -- 10.20.193.27:0/669268653 shutdown_connections 2026-02-13T02:22:29.144 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.141+0000 7f8ec4ad06c0 1 -- 10.20.193.27:0/669268653 wait complete. 2026-02-13T02:22:29.145 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.142+0000 7f8ec4ad06c0 1 Processor -- start 2026-02-13T02:22:29.145 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.142+0000 7f8ec4ad06c0 1 -- start start 2026-02-13T02:22:29.145 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.142+0000 7f8ec4ad06c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8ec010f200 0x7f8ec011d1d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:29.146 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.142+0000 7f8ec4ad06c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8ec0115410 0x7f8ec011d710 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:29.146 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.142+0000 7f8ec4ad06c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8ec0122ad0 0x7f8ec0122f70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:29.146 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.142+0000 7f8ec4ad06c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec01234e0 con 0x7f8ec0115410 2026-02-13T02:22:29.146 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.142+0000 7f8ec4ad06c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec0123650 con 0x7f8ec010f200 2026-02-13T02:22:29.146 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.142+0000 7f8ec4ad06c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec0123810 con 0x7f8ec0122ad0 2026-02-13T02:22:29.146 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.142+0000 7f8ebdd746c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8ec0115410 0x7f8ec011d710 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:29.146 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.142+0000 7f8ebdd746c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8ec0115410 0x7f8ec011d710 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:53828/0 (socket says 10.20.193.27:53828) 2026-02-13T02:22:29.146 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.142+0000 7f8ebdd746c0 1 -- 10.20.193.27:0/3594651942 learned_addr learned my addr 10.20.193.27:0/3594651942 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:29.146 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.142+0000 7f8ebed766c0 1 --2- 10.20.193.27:0/3594651942 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8ec0122ad0 0x7f8ec0122f70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:29.146 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.142+0000 7f8ebe5756c0 1 --2- 10.20.193.27:0/3594651942 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8ec010f200 0x7f8ec011d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:29.146 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.142+0000 7f8ebdd746c0 1 -- 10.20.193.27:0/3594651942 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8ec0122ad0 msgr2=0x7f8ec0122f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:29.147 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.142+0000 7f8ebdd746c0 1 --2- 10.20.193.27:0/3594651942 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8ec0122ad0 0x7f8ec0122f70 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.147 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.142+0000 7f8ebdd746c0 1 -- 10.20.193.27:0/3594651942 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8ec010f200 msgr2=0x7f8ec011d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:29.147 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.142+0000 7f8ebdd746c0 1 --2- 10.20.193.27:0/3594651942 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8ec010f200 0x7f8ec011d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.147 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.142+0000 7f8ebdd746c0 1 -- 10.20.193.27:0/3594651942 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb000a660 con 0x7f8ec0115410 2026-02-13T02:22:29.147 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.142+0000 7f8ebdd746c0 1 --2- 10.20.193.27:0/3594651942 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8ec0115410 0x7f8ec011d710 secure :-1 s=READY pgs=315 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8ea800b6c0 tx=0x7f8ea800bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:29.147 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.142+0000 7f8ea77fe6c0 1 -- 10.20.193.27:0/3594651942 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8ea80044a0 con 0x7f8ec0115410 2026-02-13T02:22:29.147 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.142+0000 7f8ea77fe6c0 1 -- 10.20.193.27:0/3594651942 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f8ea8009030 con 0x7f8ec0115410 2026-02-13T02:22:29.147 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.142+0000 7f8ea77fe6c0 1 -- 10.20.193.27:0/3594651942 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8ea800cc10 con 0x7f8ec0115410 2026-02-13T02:22:29.147 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.142+0000 7f8ec4ad06c0 1 -- 10.20.193.27:0/3594651942 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ec0163cc0 con 0x7f8ec0115410 2026-02-13T02:22:29.147 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.142+0000 7f8ec4ad06c0 1 -- 10.20.193.27:0/3594651942 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ec0164150 con 0x7f8ec0115410 2026-02-13T02:22:29.147 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.143+0000 7f8ec4ad06c0 1 -- 10.20.193.27:0/3594651942 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ec01104d0 con 0x7f8ec0115410 2026-02-13T02:22:29.147 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.144+0000 7f8ea77fe6c0 1 -- 10.20.193.27:0/3594651942 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f8ea80026b0 con 0x7f8ec0115410 2026-02-13T02:22:29.148 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.145+0000 7f8ea77fe6c0 1 --2- 10.20.193.27:0/3594651942 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8e980b5710 0x7f8e980d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:29.148 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.145+0000 7f8ebe5756c0 1 --2- 10.20.193.27:0/3594651942 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8e980b5710 0x7f8e980d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:29.149 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.145+0000 7f8ea77fe6c0 1 -- 10.20.193.27:0/3594651942 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f8ea80e3970 con 0x7f8ec0115410 2026-02-13T02:22:29.149 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.145+0000 7f8ebe5756c0 1 --2- 10.20.193.27:0/3594651942 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8e980b5710 0x7f8e980d5bc0 secure :-1 s=READY pgs=220 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8eb4004750 tx=0x7f8eb4009260 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:29.149 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.146+0000 7f8ea77fe6c0 1 -- 10.20.193.27:0/3594651942 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f8ea80a6390 con 0x7f8ec0115410 2026-02-13T02:22:29.152 INFO:tasks.cephadm.ceph_manager.ceph:need seq 42949672983 got 42949672983 for osd.1 2026-02-13T02:22:29.152 DEBUG:teuthology.parallel:result is None 2026-02-13T02:22:29.164 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913789967 got 158913789968 for osd.6 2026-02-13T02:22:29.164 DEBUG:teuthology.parallel:result is None 2026-02-13T02:22:29.175 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018897 got 128849018897 for osd.5 2026-02-13T02:22:29.175 DEBUG:teuthology.parallel:result is None 2026-02-13T02:22:29.239 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.238+0000 7f8ec4ad06c0 1 -- 10.20.193.27:0/3594651942 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f8ec010f5d0 con 0x7f8ec0115410 2026-02-13T02:22:29.240 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.238+0000 7f8ea77fe6c0 1 -- 10.20.193.27:0/3594651942 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f8ea80a9540 con 0x7f8ec0115410 2026-02-13T02:22:29.240 INFO:teuthology.orchestra.run.trial027.stdout:249108103177 2026-02-13T02:22:29.241 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.239+0000 7f8ec4ad06c0 1 -- 10.20.193.27:0/3594651942 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8e980b5710 msgr2=0x7f8e980d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:29.241 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.239+0000 7f8ec4ad06c0 1 --2- 10.20.193.27:0/3594651942 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8e980b5710 0x7f8e980d5bc0 secure :-1 s=READY pgs=220 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8eb4004750 tx=0x7f8eb4009260 comp rx=0 tx=0).stop 2026-02-13T02:22:29.241 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.239+0000 7f8ec4ad06c0 1 -- 10.20.193.27:0/3594651942 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8ec0115410 msgr2=0x7f8ec011d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:29.242 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.239+0000 7f8ec4ad06c0 1 --2- 10.20.193.27:0/3594651942 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8ec0115410 0x7f8ec011d710 secure :-1 s=READY pgs=315 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8ea800b6c0 tx=0x7f8ea800bb90 comp rx=0 tx=0).stop 2026-02-13T02:22:29.242 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.239+0000 7f8ec4ad06c0 1 -- 10.20.193.27:0/3594651942 shutdown_connections 2026-02-13T02:22:29.242 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.239+0000 7f8ec4ad06c0 1 --2- 10.20.193.27:0/3594651942 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8e980b5710 0x7f8e980d5bc0 unknown :-1 s=CLOSED pgs=220 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.242 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.239+0000 7f8ec4ad06c0 1 --2- 10.20.193.27:0/3594651942 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8ec0122ad0 0x7f8ec0122f70 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.242 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.239+0000 7f8ec4ad06c0 1 --2- 10.20.193.27:0/3594651942 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8ec0115410 0x7f8ec011d710 unknown :-1 s=CLOSED pgs=315 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.242 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.239+0000 7f8ec4ad06c0 1 --2- 10.20.193.27:0/3594651942 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8ec010f200 0x7f8ec011d1d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:29.242 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.239+0000 7f8ec4ad06c0 1 -- 10.20.193.27:0/3594651942 >> 10.20.193.27:0/3594651942 conn(0x7f8ec00ecb00 msgr2=0x7f8ec010cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:29.242 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.239+0000 7f8ec4ad06c0 1 -- 10.20.193.27:0/3594651942 shutdown_connections 2026-02-13T02:22:29.242 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:29.240+0000 7f8ec4ad06c0 1 -- 10.20.193.27:0/3594651942 wait complete. 2026-02-13T02:22:29.282 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103177 got 249108103177 for osd.10 2026-02-13T02:22:29.282 DEBUG:teuthology.parallel:result is None 2026-02-13T02:22:29.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:29 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1542532650' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2026-02-13T02:22:29.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:29 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2104342111' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2026-02-13T02:22:29.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:29 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2227139836' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2026-02-13T02:22:29.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:29 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2594122028' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2026-02-13T02:22:29.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:29 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/742511478' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2026-02-13T02:22:29.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:29 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1952203966' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2026-02-13T02:22:29.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:29 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1981214267' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2026-02-13T02:22:29.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:29 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/782565218' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2026-02-13T02:22:29.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:29 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1542532650' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2026-02-13T02:22:29.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:29 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2104342111' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2026-02-13T02:22:29.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:29 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2227139836' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2026-02-13T02:22:29.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:29 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2594122028' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2026-02-13T02:22:29.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:29 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/742511478' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2026-02-13T02:22:29.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:29 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1952203966' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2026-02-13T02:22:29.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:29 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1981214267' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2026-02-13T02:22:29.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:29 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/782565218' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2026-02-13T02:22:29.988 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.0 2026-02-13T02:22:29.988 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.2 2026-02-13T02:22:30.027 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.4 2026-02-13T02:22:30.039 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.11 2026-02-13T02:22:30.081 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.3 2026-02-13T02:22:30.083 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.8 2026-02-13T02:22:30.087 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.7 2026-02-13T02:22:30.100 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph osd last-stat-seq osd.9 2026-02-13T02:22:30.108 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:30.109 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:30.152 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:30.200 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:30.230 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:30.232 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:30.237 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:30.238 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:30.247 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.244+0000 7fbccfdaf6c0 1 -- 10.20.193.27:0/1669316152 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fbcc810fc20 msgr2=0x7fbcc8114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.247 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.244+0000 7fbccfdaf6c0 1 --2- 10.20.193.27:0/1669316152 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fbcc810fc20 0x7fbcc8114c60 secure :-1 s=READY pgs=316 gs=1 cs=0 l=1 c_cookie=10541712063755147917 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbcbc0099a0 tx=0x7fbcbc02ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:30.247 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.244+0000 7fbccfdaf6c0 1 -- 10.20.193.27:0/1669316152 shutdown_connections 2026-02-13T02:22:30.247 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.244+0000 7fbccfdaf6c0 1 --2- 10.20.193.27:0/1669316152 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fbcc8115440 0x7fbcc8175de0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.247 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.244+0000 7fbccfdaf6c0 1 --2- 10.20.193.27:0/1669316152 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fbcc810fc20 0x7fbcc8114c60 unknown :-1 s=CLOSED pgs=316 gs=1 cs=0 l=1 c_cookie=10541712063755147917 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.248 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.244+0000 7fbccfdaf6c0 1 --2- 10.20.193.27:0/1669316152 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbcc810f230 0x7fbcc810f600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.248 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.244+0000 7fbccfdaf6c0 1 -- 10.20.193.27:0/1669316152 >> 10.20.193.27:0/1669316152 conn(0x7fbcc80ecb30 msgr2=0x7fbcc810cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:30.248 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.244+0000 7fbccfdaf6c0 1 -- 10.20.193.27:0/1669316152 shutdown_connections 2026-02-13T02:22:30.248 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.244+0000 7fbccfdaf6c0 1 -- 10.20.193.27:0/1669316152 wait complete. 2026-02-13T02:22:30.248 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.244+0000 7fbccfdaf6c0 1 Processor -- start 2026-02-13T02:22:30.248 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.244+0000 7fbccfdaf6c0 1 -- start start 2026-02-13T02:22:30.248 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.244+0000 7fbccfdaf6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fbcc810f230 0x7fbcc80f4ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.248 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.244+0000 7fbccfdaf6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbcc8115440 0x7fbcc8175ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.248 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.244+0000 7fbccfdaf6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fbcc80f5310 0x7fbcc80fa590 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.248 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.244+0000 7fbccfdaf6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc80f5a00 con 0x7fbcc810f230 2026-02-13T02:22:30.249 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.244+0000 7fbccfdaf6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc80faad0 con 0x7fbcc8115440 2026-02-13T02:22:30.249 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.244+0000 7fbccfdaf6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc80fac90 con 0x7fbcc80f5310 2026-02-13T02:22:30.249 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.244+0000 7fbccdb206c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fbcc810f230 0x7fbcc80f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.249 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.244+0000 7fbccdb206c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fbcc810f230 0x7fbcc80f4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:53832/0 (socket says 10.20.193.27:53832) 2026-02-13T02:22:30.249 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.244+0000 7fbccdb206c0 1 -- 10.20.193.27:0/213258991 learned_addr learned my addr 10.20.193.27:0/213258991 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:30.249 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.245+0000 7fbcce3216c0 1 --2- 10.20.193.27:0/213258991 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fbcc80f5310 0x7fbcc80fa590 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.249 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.245+0000 7fbccd31f6c0 1 --2- 10.20.193.27:0/213258991 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbcc8115440 0x7fbcc8175ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.249 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.245+0000 7fbccdb206c0 1 -- 10.20.193.27:0/213258991 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fbcc80f5310 msgr2=0x7fbcc80fa590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.249 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.245+0000 7fbccdb206c0 1 --2- 10.20.193.27:0/213258991 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fbcc80f5310 0x7fbcc80fa590 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.249 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.245+0000 7fbccdb206c0 1 -- 10.20.193.27:0/213258991 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbcc8115440 msgr2=0x7fbcc8175ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.249 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.245+0000 7fbccdb206c0 1 --2- 10.20.193.27:0/213258991 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbcc8115440 0x7fbcc8175ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.249 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.245+0000 7fbccdb206c0 1 -- 10.20.193.27:0/213258991 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcbc009590 con 0x7fbcc810f230 2026-02-13T02:22:30.250 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.245+0000 7fbcce3216c0 1 --2- 10.20.193.27:0/213258991 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fbcc80f5310 0x7fbcc80fa590 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:22:30.250 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.245+0000 7fbccd31f6c0 1 --2- 10.20.193.27:0/213258991 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbcc8115440 0x7fbcc8175ce0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:22:30.250 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.245+0000 7fbccdb206c0 1 --2- 10.20.193.27:0/213258991 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fbcc810f230 0x7fbcc80f4ce0 secure :-1 s=READY pgs=317 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbcb800b710 tx=0x7fbcb800bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:30.250 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.245+0000 7fbcb6ffd6c0 1 -- 10.20.193.27:0/213258991 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fbcb8004250 con 0x7fbcc810f230 2026-02-13T02:22:30.250 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.245+0000 7fbccfdaf6c0 1 -- 10.20.193.27:0/213258991 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc80faf70 con 0x7fbcc810f230 2026-02-13T02:22:30.250 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.245+0000 7fbcb6ffd6c0 1 -- 10.20.193.27:0/213258991 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fbcb8009070 con 0x7fbcc810f230 2026-02-13T02:22:30.250 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.245+0000 7fbcb6ffd6c0 1 -- 10.20.193.27:0/213258991 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fbcb800ccd0 con 0x7fbcc810f230 2026-02-13T02:22:30.250 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.245+0000 7fbccfdaf6c0 1 -- 10.20.193.27:0/213258991 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc80fb700 con 0x7fbcc810f230 2026-02-13T02:22:30.250 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.245+0000 7fbccfdaf6c0 1 -- 10.20.193.27:0/213258991 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcc8110500 con 0x7fbcc810f230 2026-02-13T02:22:30.252 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.246+0000 7f43279976c0 1 -- 10.20.193.27:0/4031258 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f43200622d0 msgr2=0x7f43200607a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.252 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.246+0000 7f43279976c0 1 --2- 10.20.193.27:0/4031258 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f43200622d0 0x7f43200607a0 secure :-1 s=READY pgs=318 gs=1 cs=0 l=1 c_cookie=1047702611597122352 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4310009970 tx=0x7f431002ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:30.252 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.246+0000 7f43279976c0 1 -- 10.20.193.27:0/4031258 shutdown_connections 2026-02-13T02:22:30.252 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.246+0000 7f43279976c0 1 --2- 10.20.193.27:0/4031258 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f4320159af0 0x7f4320179ed0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.252 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.246+0000 7f43279976c0 1 --2- 10.20.193.27:0/4031258 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4320060ce0 0x7f4320061110 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.252 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.246+0000 7f43279976c0 1 --2- 10.20.193.27:0/4031258 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f43200622d0 0x7f43200607a0 unknown :-1 s=CLOSED pgs=318 gs=1 cs=0 l=1 c_cookie=1047702611597122352 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.252 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.246+0000 7f43279976c0 1 -- 10.20.193.27:0/4031258 >> 10.20.193.27:0/4031258 conn(0x7f43200ea8b0 msgr2=0x7f432010acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:30.253 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.246+0000 7f43279976c0 1 -- 10.20.193.27:0/4031258 shutdown_connections 2026-02-13T02:22:30.253 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.246+0000 7f43279976c0 1 -- 10.20.193.27:0/4031258 wait complete. 2026-02-13T02:22:30.253 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.247+0000 7f43279976c0 1 Processor -- start 2026-02-13T02:22:30.253 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.247+0000 7f43279976c0 1 -- start start 2026-02-13T02:22:30.253 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.247+0000 7f43279976c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4320060ce0 0x7f43200f23c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.253 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.247+0000 7f43279976c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f43200622d0 0x7f43200f2900 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.253 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.247+0000 7f43279976c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f4320159af0 0x7f43200f7cc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.253 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.247+0000 7f43279976c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f43200f8310 con 0x7f4320060ce0 2026-02-13T02:22:30.253 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.247+0000 7f43279976c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f43200f8480 con 0x7f43200622d0 2026-02-13T02:22:30.253 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.247+0000 7f43279976c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f43200f8640 con 0x7f4320159af0 2026-02-13T02:22:30.253 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.247+0000 7f43257086c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4320060ce0 0x7f43200f23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.253 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.247+0000 7f43257086c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4320060ce0 0x7f43200f23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:53862/0 (socket says 10.20.193.27:53862) 2026-02-13T02:22:30.254 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.247+0000 7f43257086c0 1 -- 10.20.193.27:0/953242410 learned_addr learned my addr 10.20.193.27:0/953242410 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:30.254 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.247+0000 7f4325f096c0 1 --2- 10.20.193.27:0/953242410 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f4320159af0 0x7f43200f7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.254 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.247+0000 7f43257086c0 1 -- 10.20.193.27:0/953242410 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f4320159af0 msgr2=0x7f43200f7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.254 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.247+0000 7f43257086c0 1 --2- 10.20.193.27:0/953242410 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f4320159af0 0x7f43200f7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.254 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.247+0000 7f43257086c0 1 -- 10.20.193.27:0/953242410 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f43200622d0 msgr2=0x7f43200f2900 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-13T02:22:30.254 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.247+0000 7f43257086c0 1 --2- 10.20.193.27:0/953242410 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f43200622d0 0x7f43200f2900 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.254 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.247+0000 7f43257086c0 1 -- 10.20.193.27:0/953242410 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4310009590 con 0x7f4320060ce0 2026-02-13T02:22:30.254 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.247+0000 7f43257086c0 1 --2- 10.20.193.27:0/953242410 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4320060ce0 0x7f43200f23c0 secure :-1 s=READY pgs=319 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f43100092b0 tx=0x7f431002fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:30.255 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.247+0000 7fbcb6ffd6c0 1 -- 10.20.193.27:0/213258991 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7fbcb80028c0 con 0x7fbcc810f230 2026-02-13T02:22:30.255 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.247+0000 7fbcb6ffd6c0 1 --2- 10.20.193.27:0/213258991 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fbca40b5710 0x7fbca40d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.255 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.247+0000 7fbcb6ffd6c0 1 -- 10.20.193.27:0/213258991 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7fbcb80e3be0 con 0x7fbcc810f230 2026-02-13T02:22:30.255 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.248+0000 7fbccd31f6c0 1 --2- 10.20.193.27:0/213258991 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fbca40b5710 0x7fbca40d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.255 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.248+0000 7fbccd31f6c0 1 --2- 10.20.193.27:0/213258991 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fbca40b5710 0x7fbca40d5bc0 secure :-1 s=READY pgs=221 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbcbc0023e0 tx=0x7fbcbc005b00 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:30.256 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.248+0000 7f430e7fc6c0 1 -- 10.20.193.27:0/953242410 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4310002ba0 con 0x7f4320060ce0 2026-02-13T02:22:30.256 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.248+0000 7f430e7fc6c0 1 -- 10.20.193.27:0/953242410 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4310002d00 con 0x7f4320060ce0 2026-02-13T02:22:30.256 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.248+0000 7f430e7fc6c0 1 -- 10.20.193.27:0/953242410 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f43100372f0 con 0x7f4320060ce0 2026-02-13T02:22:30.256 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.248+0000 7f43279976c0 1 -- 10.20.193.27:0/953242410 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43200f88c0 con 0x7f4320060ce0 2026-02-13T02:22:30.256 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.248+0000 7fbcb6ffd6c0 1 -- 10.20.193.27:0/213258991 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fbcb80a6640 con 0x7fbcc810f230 2026-02-13T02:22:30.257 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.248+0000 7f43279976c0 1 -- 10.20.193.27:0/953242410 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43200f8b40 con 0x7f4320060ce0 2026-02-13T02:22:30.257 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.249+0000 7f43279976c0 1 -- 10.20.193.27:0/953242410 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4320061cb0 con 0x7f4320060ce0 2026-02-13T02:22:30.258 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.250+0000 7f430e7fc6c0 1 -- 10.20.193.27:0/953242410 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f43100078f0 con 0x7f4320060ce0 2026-02-13T02:22:30.258 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.251+0000 7f430e7fc6c0 1 --2- 10.20.193.27:0/953242410 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f42f40b5710 0x7f42f40d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.258 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.251+0000 7f4324f076c0 1 --2- 10.20.193.27:0/953242410 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f42f40b5710 0x7f42f40d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.258 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.251+0000 7f430e7fc6c0 1 -- 10.20.193.27:0/953242410 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f4310107b00 con 0x7f4320060ce0 2026-02-13T02:22:30.258 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.251+0000 7f4324f076c0 1 --2- 10.20.193.27:0/953242410 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f42f40b5710 0x7f42f40d5bc0 secure :-1 s=READY pgs=222 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f43200f8d50 tx=0x7f43140093b0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:30.258 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.251+0000 7f430e7fc6c0 1 -- 10.20.193.27:0/953242410 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f43100ca5f0 con 0x7f4320060ce0 2026-02-13T02:22:30.266 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:30 trial027 ceph-mon[24567]: pgmap v190: 97 pgs: 97 active+clean; 629 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 6.2 KiB/s rd, 7.4 KiB/s wr, 8 op/s 2026-02-13T02:22:30.267 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:30 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3158728324' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2026-02-13T02:22:30.267 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:30 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2744320861' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2026-02-13T02:22:30.267 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:30 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3533716426' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2026-02-13T02:22:30.267 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:30 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3594651942' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2026-02-13T02:22:30.269 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.265+0000 7ffa45b996c0 1 -- 10.20.193.27:0/876535683 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ffa40113190 msgr2=0x7ffa40173b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.269 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.265+0000 7ffa45b996c0 1 --2- 10.20.193.27:0/876535683 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ffa40113190 0x7ffa40173b30 secure :-1 s=READY pgs=59 gs=2 cs=0 l=1 c_cookie=6103669833124757308 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffa3400a9b0 tx=0x7ffa3402fd70 comp rx=0 tx=0).stop 2026-02-13T02:22:30.269 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.266+0000 7ffa45b996c0 1 -- 10.20.193.27:0/876535683 shutdown_connections 2026-02-13T02:22:30.269 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.266+0000 7ffa45b996c0 1 --2- 10.20.193.27:0/876535683 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ffa40113190 0x7ffa40173b30 unknown :-1 s=CLOSED pgs=59 gs=2 cs=0 l=1 c_cookie=6103669833124757308 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.269 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.266+0000 7ffa45b996c0 1 --2- 10.20.193.27:0/876535683 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffa4010d970 0x7ffa401129b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.270 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.266+0000 7ffa45b996c0 1 --2- 10.20.193.27:0/876535683 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ffa4010cf80 0x7ffa4010d350 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.270 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.266+0000 7ffa45b996c0 1 -- 10.20.193.27:0/876535683 >> 10.20.193.27:0/876535683 conn(0x7ffa400ea880 msgr2=0x7ffa4010ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:30.270 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.266+0000 7ffa45b996c0 1 -- 10.20.193.27:0/876535683 shutdown_connections 2026-02-13T02:22:30.270 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.266+0000 7ffa45b996c0 1 -- 10.20.193.27:0/876535683 wait complete. 2026-02-13T02:22:30.270 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.266+0000 7ffa45b996c0 1 Processor -- start 2026-02-13T02:22:30.270 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.266+0000 7ffa45b996c0 1 -- start start 2026-02-13T02:22:30.270 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.266+0000 7ffa45b996c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ffa4010cf80 0x7ffa400f4bd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.270 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.266+0000 7ffa45b996c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffa40113190 0x7ffa40173a30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.270 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.266+0000 7ffa3f7fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ffa4010cf80 0x7ffa400f4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.270 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.266+0000 7ffa3f7fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ffa4010cf80 0x7ffa400f4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:53886/0 (socket says 10.20.193.27:53886) 2026-02-13T02:22:30.270 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.266+0000 7ffa45b996c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ffa400f5200 0x7ffa400fa4d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.271 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.266+0000 7ffa45b996c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7ffa400f58f0 con 0x7ffa4010cf80 2026-02-13T02:22:30.271 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.266+0000 7ffa45b996c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7ffa400faa10 con 0x7ffa40113190 2026-02-13T02:22:30.271 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.266+0000 7ffa45b996c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7ffa400fabd0 con 0x7ffa400f5200 2026-02-13T02:22:30.271 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.266+0000 7ffa3f7fe6c0 1 -- 10.20.193.27:0/4036026419 learned_addr learned my addr 10.20.193.27:0/4036026419 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:30.271 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.266+0000 7ffa3effd6c0 1 --2- 10.20.193.27:0/4036026419 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffa40113190 0x7ffa40173a30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.271 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.267+0000 7ffa3f7fe6c0 1 -- 10.20.193.27:0/4036026419 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ffa400f5200 msgr2=0x7ffa400fa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.271 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.267+0000 7ffa3f7fe6c0 1 --2- 10.20.193.27:0/4036026419 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ffa400f5200 0x7ffa400fa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.271 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.267+0000 7ffa3f7fe6c0 1 -- 10.20.193.27:0/4036026419 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffa40113190 msgr2=0x7ffa40173a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.271 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.267+0000 7ffa3f7fe6c0 1 --2- 10.20.193.27:0/4036026419 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffa40113190 0x7ffa40173a30 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.271 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.267+0000 7ffa3f7fe6c0 1 -- 10.20.193.27:0/4036026419 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa2c009590 con 0x7ffa4010cf80 2026-02-13T02:22:30.271 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.267+0000 7ffa3f7fe6c0 1 --2- 10.20.193.27:0/4036026419 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ffa4010cf80 0x7ffa400f4bd0 secure :-1 s=READY pgs=320 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffa2c004750 tx=0x7ffa2c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:30.271 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.267+0000 7ffa3cff96c0 1 -- 10.20.193.27:0/4036026419 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ffa2c0044a0 con 0x7ffa4010cf80 2026-02-13T02:22:30.272 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.267+0000 7ffa45b996c0 1 -- 10.20.193.27:0/4036026419 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa3400a5b0 con 0x7ffa4010cf80 2026-02-13T02:22:30.272 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.267+0000 7ffa3cff96c0 1 -- 10.20.193.27:0/4036026419 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ffa2c009030 con 0x7ffa4010cf80 2026-02-13T02:22:30.272 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.267+0000 7ffa3cff96c0 1 -- 10.20.193.27:0/4036026419 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ffa2c0071c0 con 0x7ffa4010cf80 2026-02-13T02:22:30.272 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.267+0000 7ffa45b996c0 1 -- 10.20.193.27:0/4036026419 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa400fb250 con 0x7ffa4010cf80 2026-02-13T02:22:30.272 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.268+0000 7ffa45b996c0 1 -- 10.20.193.27:0/4036026419 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa4010e250 con 0x7ffa4010cf80 2026-02-13T02:22:30.273 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.270+0000 7ffa3cff96c0 1 -- 10.20.193.27:0/4036026419 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7ffa2c0026b0 con 0x7ffa4010cf80 2026-02-13T02:22:30.273 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.270+0000 7ffa3cff96c0 1 --2- 10.20.193.27:0/4036026419 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ffa140b5710 0x7ffa140d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.273 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.271+0000 7ffa3cff96c0 1 -- 10.20.193.27:0/4036026419 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7ffa2c0e3cc0 con 0x7ffa4010cf80 2026-02-13T02:22:30.273 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.271+0000 7ffa3effd6c0 1 --2- 10.20.193.27:0/4036026419 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ffa140b5710 0x7ffa140d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.273 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.271+0000 7ffa3effd6c0 1 --2- 10.20.193.27:0/4036026419 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ffa140b5710 0x7ffa140d5bc0 secure :-1 s=READY pgs=223 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffa30005980 tx=0x7ffa30005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:30.273 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.271+0000 7ffa3cff96c0 1 -- 10.20.193.27:0/4036026419 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7ffa2c0a67b0 con 0x7ffa4010cf80 2026-02-13T02:22:30.334 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.331+0000 7fc887a1f6c0 1 -- 10.20.193.27:0/3264952445 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc880115410 msgr2=0x7fc880175db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.334 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.331+0000 7fc887a1f6c0 1 --2- 10.20.193.27:0/3264952445 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc880115410 0x7fc880175db0 secure :-1 s=READY pgs=321 gs=1 cs=0 l=1 c_cookie=2908667873223532838 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc87c00aa70 tx=0x7fc87c02fde0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.334 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.331+0000 7fc887a1f6c0 1 -- 10.20.193.27:0/3264952445 shutdown_connections 2026-02-13T02:22:30.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.331+0000 7fc887a1f6c0 1 --2- 10.20.193.27:0/3264952445 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc880115410 0x7fc880175db0 unknown :-1 s=CLOSED pgs=321 gs=1 cs=0 l=1 c_cookie=2908667873223532838 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.331+0000 7fc887a1f6c0 1 --2- 10.20.193.27:0/3264952445 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc88010fbf0 0x7fc880114c30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.331+0000 7fc887a1f6c0 1 --2- 10.20.193.27:0/3264952445 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc88010f200 0x7fc88010f5d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.331+0000 7fc887a1f6c0 1 -- 10.20.193.27:0/3264952445 >> 10.20.193.27:0/3264952445 conn(0x7fc8800ecb00 msgr2=0x7fc88010cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:30.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.331+0000 7fc887a1f6c0 1 -- 10.20.193.27:0/3264952445 shutdown_connections 2026-02-13T02:22:30.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.331+0000 7fc887a1f6c0 1 -- 10.20.193.27:0/3264952445 wait complete. 2026-02-13T02:22:30.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.331+0000 7fc887a1f6c0 1 Processor -- start 2026-02-13T02:22:30.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.331+0000 7fc887a1f6c0 1 -- start start 2026-02-13T02:22:30.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.332+0000 7fc887a1f6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc88010f200 0x7fc88011d250 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.332+0000 7fc887a1f6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc880115410 0x7fc88011d790 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.332+0000 7fc887a1f6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc880122b50 0x7fc880122ff0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.336 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.332+0000 7fc887a1f6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc880123560 con 0x7fc880115410 2026-02-13T02:22:30.336 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.332+0000 7fc887a1f6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc8801236d0 con 0x7fc880122b50 2026-02-13T02:22:30.336 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.332+0000 7fc887a1f6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc880123890 con 0x7fc88010f200 2026-02-13T02:22:30.336 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.332+0000 7fc884f8f6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc880115410 0x7fc88011d790 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.336 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.332+0000 7fc884f8f6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc880115410 0x7fc88011d790 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:53914/0 (socket says 10.20.193.27:53914) 2026-02-13T02:22:30.336 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.332+0000 7fc884f8f6c0 1 -- 10.20.193.27:0/2385519176 learned_addr learned my addr 10.20.193.27:0/2385519176 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:30.336 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.332+0000 7fc8857906c0 1 --2- 10.20.193.27:0/2385519176 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc88010f200 0x7fc88011d250 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.336 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.332+0000 7fc885f916c0 1 --2- 10.20.193.27:0/2385519176 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc880122b50 0x7fc880122ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.336 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.332+0000 7fc884f8f6c0 1 -- 10.20.193.27:0/2385519176 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc88010f200 msgr2=0x7fc88011d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.336 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.332+0000 7fc884f8f6c0 1 --2- 10.20.193.27:0/2385519176 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc88010f200 0x7fc88011d250 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.336 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.332+0000 7fc884f8f6c0 1 -- 10.20.193.27:0/2385519176 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc880122b50 msgr2=0x7fc880122ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.337 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.332+0000 7fc884f8f6c0 1 --2- 10.20.193.27:0/2385519176 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc880122b50 0x7fc880122ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.337 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.332+0000 7fc884f8f6c0 1 -- 10.20.193.27:0/2385519176 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc87c00a660 con 0x7fc880115410 2026-02-13T02:22:30.337 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.332+0000 7fc884f8f6c0 1 --2- 10.20.193.27:0/2385519176 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc880115410 0x7fc88011d790 secure :-1 s=READY pgs=322 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc87400b750 tx=0x7fc87400bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:30.337 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.332+0000 7fc86e7fc6c0 1 -- 10.20.193.27:0/2385519176 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc874004a80 con 0x7fc880115410 2026-02-13T02:22:30.337 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.332+0000 7fc887a1f6c0 1 -- 10.20.193.27:0/2385519176 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc880163d40 con 0x7fc880115410 2026-02-13T02:22:30.337 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.332+0000 7fc86e7fc6c0 1 -- 10.20.193.27:0/2385519176 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc874009030 con 0x7fc880115410 2026-02-13T02:22:30.337 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.332+0000 7fc86e7fc6c0 1 -- 10.20.193.27:0/2385519176 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc87400cd00 con 0x7fc880115410 2026-02-13T02:22:30.337 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.332+0000 7fc887a1f6c0 1 -- 10.20.193.27:0/2385519176 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8801641d0 con 0x7fc880115410 2026-02-13T02:22:30.337 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.333+0000 7fc887a1f6c0 1 -- 10.20.193.27:0/2385519176 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8801104d0 con 0x7fc880115410 2026-02-13T02:22:30.338 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.334+0000 7fc86e7fc6c0 1 -- 10.20.193.27:0/2385519176 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7fc874005020 con 0x7fc880115410 2026-02-13T02:22:30.338 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.335+0000 7fc86e7fc6c0 1 --2- 10.20.193.27:0/2385519176 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc85c0b5710 0x7fc85c0d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.338 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.335+0000 7fc86e7fc6c0 1 -- 10.20.193.27:0/2385519176 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7fc8740e3b90 con 0x7fc880115410 2026-02-13T02:22:30.339 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.335+0000 7fc8857906c0 1 --2- 10.20.193.27:0/2385519176 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc85c0b5710 0x7fc85c0d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.339 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.335+0000 7fc86e7fc6c0 1 -- 10.20.193.27:0/2385519176 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fc8740aa010 con 0x7fc880115410 2026-02-13T02:22:30.339 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.335+0000 7fc8857906c0 1 --2- 10.20.193.27:0/2385519176 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc85c0b5710 0x7fc85c0d5bc0 secure :-1 s=READY pgs=224 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc870004750 tx=0x7fc870009260 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:30.351 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.350+0000 7fbccfdaf6c0 1 -- 10.20.193.27:0/213258991 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fbcc810f600 con 0x7fbcc810f230 2026-02-13T02:22:30.351 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.350+0000 7fbcb6ffd6c0 1 -- 10.20.193.27:0/213258991 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbcb800ce60 con 0x7fbcc810f230 2026-02-13T02:22:30.352 INFO:teuthology.orchestra.run.trial027.stdout:55834574870 2026-02-13T02:22:30.354 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.352+0000 7fbccfdaf6c0 1 -- 10.20.193.27:0/213258991 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fbca40b5710 msgr2=0x7fbca40d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.354 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.352+0000 7fbccfdaf6c0 1 --2- 10.20.193.27:0/213258991 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fbca40b5710 0x7fbca40d5bc0 secure :-1 s=READY pgs=221 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbcbc0023e0 tx=0x7fbcbc005b00 comp rx=0 tx=0).stop 2026-02-13T02:22:30.354 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.352+0000 7fbccfdaf6c0 1 -- 10.20.193.27:0/213258991 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fbcc810f230 msgr2=0x7fbcc80f4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.354 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.352+0000 7fbccfdaf6c0 1 --2- 10.20.193.27:0/213258991 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fbcc810f230 0x7fbcc80f4ce0 secure :-1 s=READY pgs=317 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbcb800b710 tx=0x7fbcb800bbe0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.355 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.352+0000 7fbccfdaf6c0 1 -- 10.20.193.27:0/213258991 shutdown_connections 2026-02-13T02:22:30.355 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.352+0000 7fbccfdaf6c0 1 --2- 10.20.193.27:0/213258991 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fbca40b5710 0x7fbca40d5bc0 unknown :-1 s=CLOSED pgs=221 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.355 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.352+0000 7fbccfdaf6c0 1 --2- 10.20.193.27:0/213258991 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fbcc80f5310 0x7fbcc80fa590 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.355 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.352+0000 7fbccfdaf6c0 1 --2- 10.20.193.27:0/213258991 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbcc8115440 0x7fbcc8175ce0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.355 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.352+0000 7fbccfdaf6c0 1 --2- 10.20.193.27:0/213258991 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fbcc810f230 0x7fbcc80f4ce0 unknown :-1 s=CLOSED pgs=317 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.355 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.352+0000 7fbccfdaf6c0 1 -- 10.20.193.27:0/213258991 >> 10.20.193.27:0/213258991 conn(0x7fbcc80ecb30 msgr2=0x7fbcc810d970 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:30.355 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.352+0000 7fbccfdaf6c0 1 -- 10.20.193.27:0/213258991 shutdown_connections 2026-02-13T02:22:30.355 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.352+0000 7fbccfdaf6c0 1 -- 10.20.193.27:0/213258991 wait complete. 2026-02-13T02:22:30.356 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.355+0000 7f43279976c0 1 -- 10.20.193.27:0/953242410 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f4320061110 con 0x7f4320060ce0 2026-02-13T02:22:30.356 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.355+0000 7f430e7fc6c0 1 -- 10.20.193.27:0/953242410 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f43100cd710 con 0x7f4320060ce0 2026-02-13T02:22:30.356 INFO:teuthology.orchestra.run.trial027.stdout:30064771096 2026-02-13T02:22:30.358 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.357+0000 7f43279976c0 1 -- 10.20.193.27:0/953242410 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f42f40b5710 msgr2=0x7f42f40d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.359 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.357+0000 7f43279976c0 1 --2- 10.20.193.27:0/953242410 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f42f40b5710 0x7f42f40d5bc0 secure :-1 s=READY pgs=222 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f43200f8d50 tx=0x7f43140093b0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.359 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.357+0000 7f43279976c0 1 -- 10.20.193.27:0/953242410 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4320060ce0 msgr2=0x7f43200f23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.359 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.357+0000 7f6132f856c0 1 -- 10.20.193.27:0/1543603261 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f612c10f230 msgr2=0x7f612c10f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.359 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.357+0000 7f6132f856c0 1 --2- 10.20.193.27:0/1543603261 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f612c10f230 0x7f612c10f600 secure :-1 s=READY pgs=323 gs=1 cs=0 l=1 c_cookie=14907787493277748731 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f611c0099a0 tx=0x7f611c02ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:30.360 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.357+0000 7f43279976c0 1 --2- 10.20.193.27:0/953242410 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4320060ce0 0x7f43200f23c0 secure :-1 s=READY pgs=319 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f43100092b0 tx=0x7f431002fd40 comp rx=0 tx=0).stop 2026-02-13T02:22:30.360 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.357+0000 7f6132f856c0 1 -- 10.20.193.27:0/1543603261 shutdown_connections 2026-02-13T02:22:30.360 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.357+0000 7f6132f856c0 1 --2- 10.20.193.27:0/1543603261 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f612c115440 0x7f612c175de0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.360 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.357+0000 7f6132f856c0 1 --2- 10.20.193.27:0/1543603261 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f612c10fc20 0x7f612c114c60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.360 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.357+0000 7f6132f856c0 1 --2- 10.20.193.27:0/1543603261 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f612c10f230 0x7f612c10f600 unknown :-1 s=CLOSED pgs=323 gs=1 cs=0 l=1 c_cookie=14907787493277748731 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.360 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.357+0000 7f6132f856c0 1 -- 10.20.193.27:0/1543603261 >> 10.20.193.27:0/1543603261 conn(0x7f612c0ecb30 msgr2=0x7f612c10cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:30.361 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.357+0000 7f43279976c0 1 -- 10.20.193.27:0/953242410 shutdown_connections 2026-02-13T02:22:30.361 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.357+0000 7f43279976c0 1 --2- 10.20.193.27:0/953242410 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f42f40b5710 0x7f42f40d5bc0 unknown :-1 s=CLOSED pgs=222 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.361 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.357+0000 7f43279976c0 1 --2- 10.20.193.27:0/953242410 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f4320159af0 0x7f43200f7cc0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.361 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.357+0000 7f43279976c0 1 --2- 10.20.193.27:0/953242410 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f43200622d0 0x7f43200f2900 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.361 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.357+0000 7f43279976c0 1 --2- 10.20.193.27:0/953242410 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4320060ce0 0x7f43200f23c0 unknown :-1 s=CLOSED pgs=319 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.361 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.357+0000 7f6132f856c0 1 -- 10.20.193.27:0/1543603261 shutdown_connections 2026-02-13T02:22:30.362 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.357+0000 7f43279976c0 1 -- 10.20.193.27:0/953242410 >> 10.20.193.27:0/953242410 conn(0x7f43200ea8b0 msgr2=0x7f432010f080 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:30.362 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.357+0000 7f6132f856c0 1 -- 10.20.193.27:0/1543603261 wait complete. 2026-02-13T02:22:30.363 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.357+0000 7f43279976c0 1 -- 10.20.193.27:0/953242410 shutdown_connections 2026-02-13T02:22:30.363 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.357+0000 7f43279976c0 1 -- 10.20.193.27:0/953242410 wait complete. 2026-02-13T02:22:30.364 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f6132f856c0 1 Processor -- start 2026-02-13T02:22:30.364 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f6132f856c0 1 -- start start 2026-02-13T02:22:30.364 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f6132f856c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f612c10f230 0x7f612c11d270 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.364 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f6132f856c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f612c115440 0x7f612c11d7b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.364 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f6132f856c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f612c122b70 0x7f612c123010 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.364 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f6132f856c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f612c123580 con 0x7f612c122b70 2026-02-13T02:22:30.364 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f6132f856c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f612c1236f0 con 0x7f612c115440 2026-02-13T02:22:30.365 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f6132f856c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f612c1238b0 con 0x7f612c10f230 2026-02-13T02:22:30.365 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f61314f76c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f612c122b70 0x7f612c123010 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.365 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f61314f76c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f612c122b70 0x7f612c123010 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:53938/0 (socket says 10.20.193.27:53938) 2026-02-13T02:22:30.365 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f61314f76c0 1 -- 10.20.193.27:0/2319221594 learned_addr learned my addr 10.20.193.27:0/2319221594 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:30.365 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f6130cf66c0 1 --2- 10.20.193.27:0/2319221594 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f612c10f230 0x7f612c11d270 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.365 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f6123fff6c0 1 --2- 10.20.193.27:0/2319221594 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f612c115440 0x7f612c11d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.366 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f337c9606c0 1 -- 10.20.193.27:0/2787420004 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f33741131c0 msgr2=0x7f3374173b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.366 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f337c9606c0 1 --2- 10.20.193.27:0/2787420004 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f33741131c0 0x7f3374173b60 secure :-1 s=READY pgs=324 gs=1 cs=0 l=1 c_cookie=7409888744692375328 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f336800aa70 tx=0x7f336802fde0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.366 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f337c9606c0 1 -- 10.20.193.27:0/2787420004 shutdown_connections 2026-02-13T02:22:30.366 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f337c9606c0 1 --2- 10.20.193.27:0/2787420004 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f33741131c0 0x7f3374173b60 unknown :-1 s=CLOSED pgs=324 gs=1 cs=0 l=1 c_cookie=7409888744692375328 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.366 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f337c9606c0 1 --2- 10.20.193.27:0/2787420004 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f337410d9a0 0x7f33741129e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.366 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f337c9606c0 1 --2- 10.20.193.27:0/2787420004 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f337410cfb0 0x7f337410d380 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.366 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f337c9606c0 1 -- 10.20.193.27:0/2787420004 >> 10.20.193.27:0/2787420004 conn(0x7f33740ea8b0 msgr2=0x7f337410acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:30.366 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f337c9606c0 1 -- 10.20.193.27:0/2787420004 shutdown_connections 2026-02-13T02:22:30.366 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f337c9606c0 1 -- 10.20.193.27:0/2787420004 wait complete. 2026-02-13T02:22:30.367 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f61314f76c0 1 -- 10.20.193.27:0/2319221594 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f612c10f230 msgr2=0x7f612c11d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.367 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f61314f76c0 1 --2- 10.20.193.27:0/2319221594 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f612c10f230 0x7f612c11d270 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.367 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f61314f76c0 1 -- 10.20.193.27:0/2319221594 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f612c115440 msgr2=0x7f612c11d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.368 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f61314f76c0 1 --2- 10.20.193.27:0/2319221594 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f612c115440 0x7f612c11d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.368 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f61314f76c0 1 -- 10.20.193.27:0/2319221594 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f611c009590 con 0x7f612c122b70 2026-02-13T02:22:30.368 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f6123fff6c0 1 --2- 10.20.193.27:0/2319221594 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f612c115440 0x7f612c11d7b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:22:30.368 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f61314f76c0 1 --2- 10.20.193.27:0/2319221594 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f612c122b70 0x7f612c123010 secure :-1 s=READY pgs=325 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6128006ea0 tx=0x7f612800ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:30.368 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f6121ffb6c0 1 -- 10.20.193.27:0/2319221594 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6128003c40 con 0x7f612c122b70 2026-02-13T02:22:30.368 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f6121ffb6c0 1 -- 10.20.193.27:0/2319221594 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6128003da0 con 0x7f612c122b70 2026-02-13T02:22:30.368 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f6121ffb6c0 1 -- 10.20.193.27:0/2319221594 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f61280081c0 con 0x7f612c122b70 2026-02-13T02:22:30.368 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f6132f856c0 1 -- 10.20.193.27:0/2319221594 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f612c163d60 con 0x7f612c122b70 2026-02-13T02:22:30.368 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.358+0000 7f6132f856c0 1 -- 10.20.193.27:0/2319221594 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f612c164240 con 0x7f612c122b70 2026-02-13T02:22:30.370 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.359+0000 7f337c9606c0 1 Processor -- start 2026-02-13T02:22:30.370 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.359+0000 7f337c9606c0 1 -- start start 2026-02-13T02:22:30.370 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.359+0000 7f337c9606c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f337410cfb0 0x7f337411afa0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.370 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.359+0000 7f337c9606c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f33741131c0 0x7f337411b4e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.370 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.359+0000 7f337c9606c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f33741208a0 0x7f3374120d40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.370 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.359+0000 7f337c9606c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f33741212b0 con 0x7f33741131c0 2026-02-13T02:22:30.370 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.359+0000 7f337c9606c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3374121420 con 0x7f337410cfb0 2026-02-13T02:22:30.370 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.359+0000 7f337c9606c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f33741215e0 con 0x7f33741208a0 2026-02-13T02:22:30.370 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.359+0000 7f3379ed46c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f33741131c0 0x7f337411b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.370 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.359+0000 7f3379ed46c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f33741131c0 0x7f337411b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:53960/0 (socket says 10.20.193.27:53960) 2026-02-13T02:22:30.371 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.359+0000 7f3379ed46c0 1 -- 10.20.193.27:0/3563207069 learned_addr learned my addr 10.20.193.27:0/3563207069 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:30.371 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.359+0000 7f337a6d56c0 1 --2- 10.20.193.27:0/3563207069 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f337410cfb0 0x7f337411afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.371 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.359+0000 7f337aed66c0 1 --2- 10.20.193.27:0/3563207069 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f33741208a0 0x7f3374120d40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.371 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.359+0000 7f3379ed46c0 1 -- 10.20.193.27:0/3563207069 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f33741208a0 msgr2=0x7f3374120d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.371 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.359+0000 7f3379ed46c0 1 --2- 10.20.193.27:0/3563207069 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f33741208a0 0x7f3374120d40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.371 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.359+0000 7f3379ed46c0 1 -- 10.20.193.27:0/3563207069 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f337410cfb0 msgr2=0x7f337411afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.371 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.359+0000 7f3379ed46c0 1 --2- 10.20.193.27:0/3563207069 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f337410cfb0 0x7f337411afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.371 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.359+0000 7f3379ed46c0 1 -- 10.20.193.27:0/3563207069 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336800a660 con 0x7f33741131c0 2026-02-13T02:22:30.371 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.359+0000 7f337a6d56c0 1 --2- 10.20.193.27:0/3563207069 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f337410cfb0 0x7f337411afa0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:22:30.372 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.359+0000 7f6132f856c0 1 -- 10.20.193.27:0/2319221594 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f612c110500 con 0x7f612c122b70 2026-02-13T02:22:30.372 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.360+0000 7f6121ffb6c0 1 -- 10.20.193.27:0/2319221594 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f6128005860 con 0x7f612c122b70 2026-02-13T02:22:30.372 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.361+0000 7f6121ffb6c0 1 --2- 10.20.193.27:0/2319221594 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f61080b5710 0x7f61080d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.372 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.361+0000 7f3379ed46c0 1 --2- 10.20.193.27:0/3563207069 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f33741131c0 0x7f337411b4e0 secure :-1 s=READY pgs=326 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3360005dc0 tx=0x7f336000bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:30.373 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.361+0000 7f6121ffb6c0 1 -- 10.20.193.27:0/2319221594 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f61280e4be0 con 0x7f612c122b70 2026-02-13T02:22:30.373 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.361+0000 7f6130cf66c0 1 --2- 10.20.193.27:0/2319221594 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f61080b5710 0x7f61080d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.373 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.361+0000 7f6121ffb6c0 1 -- 10.20.193.27:0/2319221594 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f61280a7600 con 0x7f612c122b70 2026-02-13T02:22:30.374 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.361+0000 7f335f7fe6c0 1 -- 10.20.193.27:0/3563207069 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3360004250 con 0x7f33741131c0 2026-02-13T02:22:30.374 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.361+0000 7f337c9606c0 1 -- 10.20.193.27:0/3563207069 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3374161a90 con 0x7f33741131c0 2026-02-13T02:22:30.374 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.361+0000 7f337c9606c0 1 -- 10.20.193.27:0/3563207069 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3374161f70 con 0x7f33741131c0 2026-02-13T02:22:30.374 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.361+0000 7f6130cf66c0 1 --2- 10.20.193.27:0/2319221594 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f61080b5710 0x7f61080d5bc0 secure :-1 s=READY pgs=225 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f611c0023e0 tx=0x7f611c005c60 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:30.375 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.361+0000 7f335f7fe6c0 1 -- 10.20.193.27:0/3563207069 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3360009070 con 0x7f33741131c0 2026-02-13T02:22:30.375 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.361+0000 7f335f7fe6c0 1 -- 10.20.193.27:0/3563207069 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f336000cd50 con 0x7f33741131c0 2026-02-13T02:22:30.378 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.362+0000 7faadd8a46c0 1 -- 10.20.193.27:0/841341977 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7faad810fc20 msgr2=0x7faad8114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.378 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.362+0000 7faadd8a46c0 1 --2- 10.20.193.27:0/841341977 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7faad810fc20 0x7faad8114c60 secure :-1 s=READY pgs=327 gs=2 cs=0 l=1 c_cookie=10462482053187281803 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7faac00099a0 tx=0x7faac002ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:30.378 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.362+0000 7faadd8a46c0 1 -- 10.20.193.27:0/841341977 shutdown_connections 2026-02-13T02:22:30.378 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.362+0000 7faadd8a46c0 1 --2- 10.20.193.27:0/841341977 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faad8115440 0x7faad8175de0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.378 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.362+0000 7faadd8a46c0 1 --2- 10.20.193.27:0/841341977 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7faad810fc20 0x7faad8114c60 unknown :-1 s=CLOSED pgs=327 gs=2 cs=0 l=1 c_cookie=10462482053187281803 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.378 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.362+0000 7faadd8a46c0 1 --2- 10.20.193.27:0/841341977 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7faad810f230 0x7faad810f600 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.378 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.362+0000 7faadd8a46c0 1 -- 10.20.193.27:0/841341977 >> 10.20.193.27:0/841341977 conn(0x7faad80ecb30 msgr2=0x7faad810cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:30.378 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.362+0000 7faadd8a46c0 1 -- 10.20.193.27:0/841341977 shutdown_connections 2026-02-13T02:22:30.378 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.362+0000 7faadd8a46c0 1 -- 10.20.193.27:0/841341977 wait complete. 2026-02-13T02:22:30.379 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.362+0000 7faadd8a46c0 1 Processor -- start 2026-02-13T02:22:30.379 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.363+0000 7faadd8a46c0 1 -- start start 2026-02-13T02:22:30.379 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.363+0000 7faadd8a46c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7faad810f230 0x7faad810a3d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.379 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.363+0000 7faadd8a46c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faad8115440 0x7faad8175ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.379 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.363+0000 7faad6ffd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7faad810f230 0x7faad810a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.379 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.363+0000 7faad6ffd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7faad810f230 0x7faad810a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:53974/0 (socket says 10.20.193.27:53974) 2026-02-13T02:22:30.379 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.363+0000 7faadd8a46c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7faad810d250 0x7faad810a970 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.379 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.363+0000 7faad6ffd6c0 1 -- 10.20.193.27:0/158182147 learned_addr learned my addr 10.20.193.27:0/158182147 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:30.379 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.363+0000 7faadd8a46c0 1 -- 10.20.193.27:0/158182147 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7faad810b100 con 0x7faad810f230 2026-02-13T02:22:30.379 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.363+0000 7faadd8a46c0 1 -- 10.20.193.27:0/158182147 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7faad810b270 con 0x7faad8115440 2026-02-13T02:22:30.379 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.363+0000 7faadd8a46c0 1 -- 10.20.193.27:0/158182147 --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7faad80f1070 con 0x7faad810d250 2026-02-13T02:22:30.380 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.363+0000 7faad67fc6c0 1 --2- 10.20.193.27:0/158182147 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faad8115440 0x7faad8175ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.380 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.363+0000 7faad77fe6c0 1 --2- 10.20.193.27:0/158182147 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7faad810d250 0x7faad810a970 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.380 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.363+0000 7faad6ffd6c0 1 -- 10.20.193.27:0/158182147 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7faad810d250 msgr2=0x7faad810a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.380 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.363+0000 7faad6ffd6c0 1 --2- 10.20.193.27:0/158182147 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7faad810d250 0x7faad810a970 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.380 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.363+0000 7faad6ffd6c0 1 -- 10.20.193.27:0/158182147 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faad8115440 msgr2=0x7faad8175ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.380 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.363+0000 7faad6ffd6c0 1 --2- 10.20.193.27:0/158182147 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faad8115440 0x7faad8175ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.380 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.363+0000 7faad6ffd6c0 1 -- 10.20.193.27:0/158182147 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac0009590 con 0x7faad810f230 2026-02-13T02:22:30.380 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.363+0000 7faad6ffd6c0 1 --2- 10.20.193.27:0/158182147 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7faad810f230 0x7faad810a3d0 secure :-1 s=READY pgs=328 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7faacc00b710 tx=0x7faacc00bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:30.380 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.363+0000 7faad77fe6c0 1 --2- 10.20.193.27:0/158182147 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7faad810d250 0x7faad810a970 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:22:30.380 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.364+0000 7faadc8a26c0 1 -- 10.20.193.27:0/158182147 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7faacc004250 con 0x7faad810f230 2026-02-13T02:22:30.380 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.364+0000 7faadc8a26c0 1 -- 10.20.193.27:0/158182147 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7faacc009070 con 0x7faad810f230 2026-02-13T02:22:30.380 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.365+0000 7faadc8a26c0 1 -- 10.20.193.27:0/158182147 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7faacc00cd70 con 0x7faad810f230 2026-02-13T02:22:30.381 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.365+0000 7faadd8a46c0 1 -- 10.20.193.27:0/158182147 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faad80f1350 con 0x7faad810f230 2026-02-13T02:22:30.381 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.365+0000 7faadd8a46c0 1 -- 10.20.193.27:0/158182147 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faad80f1800 con 0x7faad810f230 2026-02-13T02:22:30.381 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.362+0000 7f337c9606c0 1 -- 10.20.193.27:0/3563207069 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f337410e280 con 0x7f33741131c0 2026-02-13T02:22:30.381 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.365+0000 7f335f7fe6c0 1 -- 10.20.193.27:0/3563207069 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f33600028c0 con 0x7f33741131c0 2026-02-13T02:22:30.382 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.366+0000 7faadd8a46c0 1 -- 10.20.193.27:0/158182147 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faad8110500 con 0x7faad810f230 2026-02-13T02:22:30.382 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.366+0000 7f335f7fe6c0 1 --2- 10.20.193.27:0/3563207069 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f33500b5710 0x7f33500d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.382 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.366+0000 7f337a6d56c0 1 --2- 10.20.193.27:0/3563207069 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f33500b5710 0x7f33500d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.382 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.366+0000 7f335f7fe6c0 1 -- 10.20.193.27:0/3563207069 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f33600e3b30 con 0x7f33741131c0 2026-02-13T02:22:30.383 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.366+0000 7f335f7fe6c0 1 -- 10.20.193.27:0/3563207069 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f336011c5e0 con 0x7f33741131c0 2026-02-13T02:22:30.383 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.366+0000 7faadc8a26c0 1 -- 10.20.193.27:0/158182147 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7faacc0043b0 con 0x7faad810f230 2026-02-13T02:22:30.383 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.366+0000 7f337a6d56c0 1 --2- 10.20.193.27:0/3563207069 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f33500b5710 0x7f33500d5bc0 secure :-1 s=READY pgs=226 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f336c004750 tx=0x7f336c009260 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:30.384 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.367+0000 7faadc8a26c0 1 --2- 10.20.193.27:0/158182147 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7faaac0b5610 0x7faaac0d5ac0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.384 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.367+0000 7faadc8a26c0 1 -- 10.20.193.27:0/158182147 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7faacc0e3a00 con 0x7faad810f230 2026-02-13T02:22:30.384 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.367+0000 7faad67fc6c0 1 --2- 10.20.193.27:0/158182147 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7faaac0b5610 0x7faaac0d5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.384 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.368+0000 7faad67fc6c0 1 --2- 10.20.193.27:0/158182147 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7faaac0b5610 0x7faaac0d5ac0 secure :-1 s=READY pgs=227 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7faac0005b40 tx=0x7faac00023a0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:30.384 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.368+0000 7faadc8a26c0 1 -- 10.20.193.27:0/158182147 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7faacc0a6420 con 0x7faad810f230 2026-02-13T02:22:30.385 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.374+0000 7ffa45b996c0 1 -- 10.20.193.27:0/4036026419 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ffa4010d970 con 0x7ffa4010cf80 2026-02-13T02:22:30.385 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.375+0000 7ffa3cff96c0 1 -- 10.20.193.27:0/4036026419 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ffa2c0a98d0 con 0x7ffa4010cf80 2026-02-13T02:22:30.385 INFO:teuthology.orchestra.run.trial027.stdout:111669149715 2026-02-13T02:22:30.386 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.378+0000 7ffa45b996c0 1 -- 10.20.193.27:0/4036026419 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ffa140b5710 msgr2=0x7ffa140d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.386 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.378+0000 7ffa45b996c0 1 --2- 10.20.193.27:0/4036026419 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ffa140b5710 0x7ffa140d5bc0 secure :-1 s=READY pgs=223 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffa30005980 tx=0x7ffa30005910 comp rx=0 tx=0).stop 2026-02-13T02:22:30.386 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.378+0000 7ffa45b996c0 1 -- 10.20.193.27:0/4036026419 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ffa4010cf80 msgr2=0x7ffa400f4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.386 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.378+0000 7ffa45b996c0 1 --2- 10.20.193.27:0/4036026419 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ffa4010cf80 0x7ffa400f4bd0 secure :-1 s=READY pgs=320 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffa2c004750 tx=0x7ffa2c00be00 comp rx=0 tx=0).stop 2026-02-13T02:22:30.386 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.378+0000 7f01714296c0 1 -- 10.20.193.27:0/3137737173 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f016c10fc20 msgr2=0x7f016c114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.386 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.378+0000 7f01714296c0 1 --2- 10.20.193.27:0/3137737173 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f016c10fc20 0x7f016c114c60 secure :-1 s=READY pgs=329 gs=1 cs=0 l=1 c_cookie=8818779592794212914 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f01540099a0 tx=0x7f015402ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:30.387 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.378+0000 7ffa45b996c0 1 -- 10.20.193.27:0/4036026419 shutdown_connections 2026-02-13T02:22:30.387 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.378+0000 7ffa45b996c0 1 --2- 10.20.193.27:0/4036026419 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ffa140b5710 0x7ffa140d5bc0 unknown :-1 s=CLOSED pgs=223 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.387 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.378+0000 7ffa45b996c0 1 --2- 10.20.193.27:0/4036026419 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ffa400f5200 0x7ffa400fa4d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.387 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.378+0000 7f01714296c0 1 -- 10.20.193.27:0/3137737173 shutdown_connections 2026-02-13T02:22:30.387 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.378+0000 7f01714296c0 1 --2- 10.20.193.27:0/3137737173 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f016c115440 0x7f016c175de0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.387 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.378+0000 7f01714296c0 1 --2- 10.20.193.27:0/3137737173 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f016c10fc20 0x7f016c114c60 unknown :-1 s=CLOSED pgs=329 gs=1 cs=0 l=1 c_cookie=8818779592794212914 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.387 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.378+0000 7f01714296c0 1 --2- 10.20.193.27:0/3137737173 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f016c10f230 0x7f016c10f600 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.387 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.378+0000 7f01714296c0 1 -- 10.20.193.27:0/3137737173 >> 10.20.193.27:0/3137737173 conn(0x7f016c0ecb30 msgr2=0x7f016c10cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:30.388 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.378+0000 7ffa45b996c0 1 --2- 10.20.193.27:0/4036026419 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffa40113190 0x7ffa40173a30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.388 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.378+0000 7ffa45b996c0 1 --2- 10.20.193.27:0/4036026419 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ffa4010cf80 0x7ffa400f4bd0 unknown :-1 s=CLOSED pgs=320 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.388 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.378+0000 7ffa45b996c0 1 -- 10.20.193.27:0/4036026419 >> 10.20.193.27:0/4036026419 conn(0x7ffa400ea880 msgr2=0x7ffa4010e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:30.388 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.378+0000 7ffa45b996c0 1 -- 10.20.193.27:0/4036026419 shutdown_connections 2026-02-13T02:22:30.388 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.378+0000 7ffa45b996c0 1 -- 10.20.193.27:0/4036026419 wait complete. 2026-02-13T02:22:30.390 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.378+0000 7f01714296c0 1 -- 10.20.193.27:0/3137737173 shutdown_connections 2026-02-13T02:22:30.390 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.378+0000 7f01714296c0 1 -- 10.20.193.27:0/3137737173 wait complete. 2026-02-13T02:22:30.391 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.378+0000 7f01714296c0 1 Processor -- start 2026-02-13T02:22:30.391 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.378+0000 7f01714296c0 1 -- start start 2026-02-13T02:22:30.391 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.378+0000 7f01714296c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f016c10f230 0x7f016c0f4c40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.391 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.378+0000 7f01714296c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f016c115440 0x7f016c175ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.391 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.378+0000 7f01714296c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f016c0f5270 0x7f016c0fa540 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.391 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.379+0000 7f01714296c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f016c0f5960 con 0x7f016c115440 2026-02-13T02:22:30.391 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.379+0000 7f01714296c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f016c0faa80 con 0x7f016c10f230 2026-02-13T02:22:30.391 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.379+0000 7f01714296c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f016c0fac40 con 0x7f016c0f5270 2026-02-13T02:22:30.391 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.379+0000 7f016affd6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f016c10f230 0x7f016c0f4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.391 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.379+0000 7f016affd6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f016c10f230 0x7f016c0f4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.27:49686/0 (socket says 10.20.193.27:49686) 2026-02-13T02:22:30.391 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.379+0000 7f016affd6c0 1 -- 10.20.193.27:0/2697967832 learned_addr learned my addr 10.20.193.27:0/2697967832 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:30.391 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.379+0000 7f016a7fc6c0 1 --2- 10.20.193.27:0/2697967832 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f016c115440 0x7f016c175ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.391 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.379+0000 7f016b7fe6c0 1 --2- 10.20.193.27:0/2697967832 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f016c0f5270 0x7f016c0fa540 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.391 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.379+0000 7f016affd6c0 1 -- 10.20.193.27:0/2697967832 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f016c0f5270 msgr2=0x7f016c0fa540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.392 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.379+0000 7f016affd6c0 1 --2- 10.20.193.27:0/2697967832 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f016c0f5270 0x7f016c0fa540 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.392 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.379+0000 7f016affd6c0 1 -- 10.20.193.27:0/2697967832 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f016c115440 msgr2=0x7f016c175ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.392 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.379+0000 7f016affd6c0 1 --2- 10.20.193.27:0/2697967832 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f016c115440 0x7f016c175ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.392 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.379+0000 7f016affd6c0 1 -- 10.20.193.27:0/2697967832 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0154009590 con 0x7f016c10f230 2026-02-13T02:22:30.392 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.379+0000 7f016affd6c0 1 --2- 10.20.193.27:0/2697967832 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f016c10f230 0x7f016c0f4c40 secure :-1 s=READY pgs=59 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f016000b6f0 tx=0x7f016000bbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:30.392 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.379+0000 7f014bfff6c0 1 -- 10.20.193.27:0/2697967832 <== mon.2 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f01600044a0 con 0x7f016c10f230 2026-02-13T02:22:30.392 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.379+0000 7f01714296c0 1 -- 10.20.193.27:0/2697967832 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f016c0faf20 con 0x7f016c10f230 2026-02-13T02:22:30.392 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.379+0000 7f014bfff6c0 1 -- 10.20.193.27:0/2697967832 <== mon.2 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f0160009030 con 0x7f016c10f230 2026-02-13T02:22:30.392 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.379+0000 7f014bfff6c0 1 -- 10.20.193.27:0/2697967832 <== mon.2 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f016000cbb0 con 0x7f016c10f230 2026-02-13T02:22:30.392 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.379+0000 7f01714296c0 1 -- 10.20.193.27:0/2697967832 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f016c0fb6b0 con 0x7f016c10f230 2026-02-13T02:22:30.392 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.380+0000 7f01714296c0 1 -- 10.20.193.27:0/2697967832 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f016c110500 con 0x7f016c10f230 2026-02-13T02:22:30.392 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.381+0000 7f014bfff6c0 1 -- 10.20.193.27:0/2697967832 <== mon.2 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f01600026b0 con 0x7f016c10f230 2026-02-13T02:22:30.392 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.382+0000 7f014bfff6c0 1 --2- 10.20.193.27:0/2697967832 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f01400b5710 0x7f01400d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.392 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.382+0000 7f016a7fc6c0 1 --2- 10.20.193.27:0/2697967832 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f01400b5710 0x7f01400d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.392 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.382+0000 7f014bfff6c0 1 -- 10.20.193.27:0/2697967832 <== mon.2 v2:10.20.193.28:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f01600e3950 con 0x7f016c10f230 2026-02-13T02:22:30.393 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.382+0000 7f016a7fc6c0 1 --2- 10.20.193.27:0/2697967832 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f01400b5710 0x7f01400d5bc0 secure :-1 s=READY pgs=228 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0154005b40 tx=0x7f01540023a0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:30.393 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.382+0000 7f014bfff6c0 1 -- 10.20.193.27:0/2697967832 <== mon.2 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f01600a63b0 con 0x7f016c10f230 2026-02-13T02:22:30.410 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574869 got 55834574870 for osd.2 2026-02-13T02:22:30.410 DEBUG:teuthology.parallel:result is None 2026-02-13T02:22:30.413 INFO:tasks.cephadm.ceph_manager.ceph:need seq 30064771096 got 30064771096 for osd.0 2026-02-13T02:22:30.413 DEBUG:teuthology.parallel:result is None 2026-02-13T02:22:30.424 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149714 got 111669149715 for osd.4 2026-02-13T02:22:30.424 DEBUG:teuthology.parallel:result is None 2026-02-13T02:22:30.444 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.442+0000 7fc887a1f6c0 1 -- 10.20.193.27:0/2385519176 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fc88011e3d0 con 0x7fc880115410 2026-02-13T02:22:30.444 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.442+0000 7fc86e7fc6c0 1 -- 10.20.193.27:0/2385519176 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fc8740e3e60 con 0x7fc880115410 2026-02-13T02:22:30.444 INFO:teuthology.orchestra.run.trial027.stdout:279172874247 2026-02-13T02:22:30.446 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.444+0000 7fc887a1f6c0 1 -- 10.20.193.27:0/2385519176 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc85c0b5710 msgr2=0x7fc85c0d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.446 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.444+0000 7fc887a1f6c0 1 --2- 10.20.193.27:0/2385519176 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc85c0b5710 0x7fc85c0d5bc0 secure :-1 s=READY pgs=224 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc870004750 tx=0x7fc870009260 comp rx=0 tx=0).stop 2026-02-13T02:22:30.446 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.444+0000 7fc887a1f6c0 1 -- 10.20.193.27:0/2385519176 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc880115410 msgr2=0x7fc88011d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.446 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.444+0000 7fc887a1f6c0 1 --2- 10.20.193.27:0/2385519176 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc880115410 0x7fc88011d790 secure :-1 s=READY pgs=322 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc87400b750 tx=0x7fc87400bc20 comp rx=0 tx=0).stop 2026-02-13T02:22:30.446 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.444+0000 7fc887a1f6c0 1 -- 10.20.193.27:0/2385519176 shutdown_connections 2026-02-13T02:22:30.446 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.444+0000 7fc887a1f6c0 1 --2- 10.20.193.27:0/2385519176 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc85c0b5710 0x7fc85c0d5bc0 unknown :-1 s=CLOSED pgs=224 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.446 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.444+0000 7fc887a1f6c0 1 --2- 10.20.193.27:0/2385519176 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc880122b50 0x7fc880122ff0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.447 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.444+0000 7fc887a1f6c0 1 --2- 10.20.193.27:0/2385519176 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc880115410 0x7fc88011d790 unknown :-1 s=CLOSED pgs=322 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.447 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.444+0000 7fc887a1f6c0 1 --2- 10.20.193.27:0/2385519176 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc88010f200 0x7fc88011d250 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.447 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.444+0000 7fc887a1f6c0 1 -- 10.20.193.27:0/2385519176 >> 10.20.193.27:0/2385519176 conn(0x7fc8800ecb00 msgr2=0x7fc88010cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:30.447 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.444+0000 7fc887a1f6c0 1 -- 10.20.193.27:0/2385519176 shutdown_connections 2026-02-13T02:22:30.447 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.444+0000 7fc887a1f6c0 1 -- 10.20.193.27:0/2385519176 wait complete. 2026-02-13T02:22:30.466 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.464+0000 7f6132f856c0 1 -- 10.20.193.27:0/2319221594 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f612c10f600 con 0x7f612c122b70 2026-02-13T02:22:30.466 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.465+0000 7f6121ffb6c0 1 -- 10.20.193.27:0/2319221594 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f61280aa7b0 con 0x7f612c122b70 2026-02-13T02:22:30.466 INFO:teuthology.orchestra.run.trial027.stdout:81604378644 2026-02-13T02:22:30.468 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.466+0000 7faadd8a46c0 1 -- 10.20.193.27:0/158182147 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7faad810f600 con 0x7faad810f230 2026-02-13T02:22:30.468 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.466+0000 7faadc8a26c0 1 -- 10.20.193.27:0/158182147 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7faacc0a95d0 con 0x7faad810f230 2026-02-13T02:22:30.468 INFO:teuthology.orchestra.run.trial027.stdout:231928233995 2026-02-13T02:22:30.469 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.467+0000 7f6132f856c0 1 -- 10.20.193.27:0/2319221594 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f61080b5710 msgr2=0x7f61080d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.469 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.467+0000 7f6132f856c0 1 --2- 10.20.193.27:0/2319221594 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f61080b5710 0x7f61080d5bc0 secure :-1 s=READY pgs=225 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f611c0023e0 tx=0x7f611c005c60 comp rx=0 tx=0).stop 2026-02-13T02:22:30.469 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.467+0000 7f6132f856c0 1 -- 10.20.193.27:0/2319221594 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f612c122b70 msgr2=0x7f612c123010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.469 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.467+0000 7f6132f856c0 1 --2- 10.20.193.27:0/2319221594 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f612c122b70 0x7f612c123010 secure :-1 s=READY pgs=325 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6128006ea0 tx=0x7f612800ce50 comp rx=0 tx=0).stop 2026-02-13T02:22:30.469 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.467+0000 7f6132f856c0 1 -- 10.20.193.27:0/2319221594 shutdown_connections 2026-02-13T02:22:30.469 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.467+0000 7f6132f856c0 1 --2- 10.20.193.27:0/2319221594 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f61080b5710 0x7f61080d5bc0 unknown :-1 s=CLOSED pgs=225 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.469 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.467+0000 7f6132f856c0 1 --2- 10.20.193.27:0/2319221594 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f612c122b70 0x7f612c123010 unknown :-1 s=CLOSED pgs=325 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.469 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.467+0000 7f6132f856c0 1 --2- 10.20.193.27:0/2319221594 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f612c115440 0x7f612c11d7b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.470 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.467+0000 7f6132f856c0 1 --2- 10.20.193.27:0/2319221594 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f612c10f230 0x7f612c11d270 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.470 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.467+0000 7f6132f856c0 1 -- 10.20.193.27:0/2319221594 >> 10.20.193.27:0/2319221594 conn(0x7f612c0ecb30 msgr2=0x7f612c10cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:30.470 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.467+0000 7f6132f856c0 1 -- 10.20.193.27:0/2319221594 shutdown_connections 2026-02-13T02:22:30.470 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.467+0000 7f6132f856c0 1 -- 10.20.193.27:0/2319221594 wait complete. 2026-02-13T02:22:30.470 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.467+0000 7f337c9606c0 1 -- 10.20.193.27:0/3563207069 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f337411c120 con 0x7f33741131c0 2026-02-13T02:22:30.470 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.467+0000 7f335f7fe6c0 1 -- 10.20.193.27:0/3563207069 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f33600a6550 con 0x7f33741131c0 2026-02-13T02:22:30.471 INFO:teuthology.orchestra.run.trial027.stdout:201863462924 2026-02-13T02:22:30.471 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.469+0000 7faadd8a46c0 1 -- 10.20.193.27:0/158182147 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7faaac0b5610 msgr2=0x7faaac0d5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.472 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.469+0000 7faadd8a46c0 1 --2- 10.20.193.27:0/158182147 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7faaac0b5610 0x7faaac0d5ac0 secure :-1 s=READY pgs=227 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7faac0005b40 tx=0x7faac00023a0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.472 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.469+0000 7faadd8a46c0 1 -- 10.20.193.27:0/158182147 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7faad810f230 msgr2=0x7faad810a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.472 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.469+0000 7faadd8a46c0 1 --2- 10.20.193.27:0/158182147 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7faad810f230 0x7faad810a3d0 secure :-1 s=READY pgs=328 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7faacc00b710 tx=0x7faacc00bbe0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.472 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.469+0000 7faadd8a46c0 1 -- 10.20.193.27:0/158182147 shutdown_connections 2026-02-13T02:22:30.472 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.469+0000 7faadd8a46c0 1 --2- 10.20.193.27:0/158182147 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7faaac0b5610 0x7faaac0d5ac0 unknown :-1 s=CLOSED pgs=227 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.472 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.469+0000 7faadd8a46c0 1 --2- 10.20.193.27:0/158182147 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7faad810d250 0x7faad810a970 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.472 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.469+0000 7faadd8a46c0 1 --2- 10.20.193.27:0/158182147 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faad8115440 0x7faad8175ce0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.472 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.469+0000 7faadd8a46c0 1 --2- 10.20.193.27:0/158182147 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7faad810f230 0x7faad810a3d0 unknown :-1 s=CLOSED pgs=328 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.472 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.469+0000 7faadd8a46c0 1 -- 10.20.193.27:0/158182147 >> 10.20.193.27:0/158182147 conn(0x7faad80ecb30 msgr2=0x7faad8113e60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:30.472 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.469+0000 7faadd8a46c0 1 -- 10.20.193.27:0/158182147 shutdown_connections 2026-02-13T02:22:30.473 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.469+0000 7faadd8a46c0 1 -- 10.20.193.27:0/158182147 wait complete. 2026-02-13T02:22:30.473 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.469+0000 7f337c9606c0 1 -- 10.20.193.27:0/3563207069 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f33500b5710 msgr2=0x7f33500d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.473 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.469+0000 7f337c9606c0 1 --2- 10.20.193.27:0/3563207069 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f33500b5710 0x7f33500d5bc0 secure :-1 s=READY pgs=226 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f336c004750 tx=0x7f336c009260 comp rx=0 tx=0).stop 2026-02-13T02:22:30.474 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.469+0000 7f337c9606c0 1 -- 10.20.193.27:0/3563207069 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f33741131c0 msgr2=0x7f337411b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.474 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.469+0000 7f337c9606c0 1 --2- 10.20.193.27:0/3563207069 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f33741131c0 0x7f337411b4e0 secure :-1 s=READY pgs=326 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3360005dc0 tx=0x7f336000bbb0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.474 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.470+0000 7f337c9606c0 1 -- 10.20.193.27:0/3563207069 shutdown_connections 2026-02-13T02:22:30.474 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.470+0000 7f337c9606c0 1 --2- 10.20.193.27:0/3563207069 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f33500b5710 0x7f33500d5bc0 unknown :-1 s=CLOSED pgs=226 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.474 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.470+0000 7f337c9606c0 1 --2- 10.20.193.27:0/3563207069 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f33741208a0 0x7f3374120d40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.474 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.470+0000 7f337c9606c0 1 --2- 10.20.193.27:0/3563207069 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f33741131c0 0x7f337411b4e0 unknown :-1 s=CLOSED pgs=326 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.474 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.470+0000 7f337c9606c0 1 --2- 10.20.193.27:0/3563207069 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f337410cfb0 0x7f337411afa0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.474 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.470+0000 7f337c9606c0 1 -- 10.20.193.27:0/3563207069 >> 10.20.193.27:0/3563207069 conn(0x7f33740ea8b0 msgr2=0x7f337410ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:30.474 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.470+0000 7f337c9606c0 1 -- 10.20.193.27:0/3563207069 shutdown_connections 2026-02-13T02:22:30.474 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.470+0000 7f337c9606c0 1 -- 10.20.193.27:0/3563207069 wait complete. 2026-02-13T02:22:30.482 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.480+0000 7f01714296c0 1 -- 10.20.193.27:0/2697967832 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f016c0f5d90 con 0x7f016c10f230 2026-02-13T02:22:30.482 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.481+0000 7f014bfff6c0 1 -- 10.20.193.27:0/2697967832 <== mon.2 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f01600a9560 con 0x7f016c10f230 2026-02-13T02:22:30.483 INFO:teuthology.orchestra.run.trial027.stdout:171798691853 2026-02-13T02:22:30.486 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.485+0000 7f01714296c0 1 -- 10.20.193.27:0/2697967832 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f01400b5710 msgr2=0x7f01400d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.486 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.485+0000 7f01714296c0 1 --2- 10.20.193.27:0/2697967832 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f01400b5710 0x7f01400d5bc0 secure :-1 s=READY pgs=228 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0154005b40 tx=0x7f01540023a0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.486 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.485+0000 7f01714296c0 1 -- 10.20.193.27:0/2697967832 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f016c10f230 msgr2=0x7f016c0f4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.487 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.485+0000 7f01714296c0 1 --2- 10.20.193.27:0/2697967832 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f016c10f230 0x7f016c0f4c40 secure :-1 s=READY pgs=59 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f016000b6f0 tx=0x7f016000bbc0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.487 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.485+0000 7f01714296c0 1 -- 10.20.193.27:0/2697967832 shutdown_connections 2026-02-13T02:22:30.487 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.485+0000 7f01714296c0 1 --2- 10.20.193.27:0/2697967832 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f01400b5710 0x7f01400d5bc0 unknown :-1 s=CLOSED pgs=228 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.487 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.485+0000 7f01714296c0 1 --2- 10.20.193.27:0/2697967832 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f016c0f5270 0x7f016c0fa540 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.487 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.485+0000 7f01714296c0 1 --2- 10.20.193.27:0/2697967832 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f016c115440 0x7f016c175ce0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.487 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.485+0000 7f01714296c0 1 --2- 10.20.193.27:0/2697967832 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f016c10f230 0x7f016c0f4c40 unknown :-1 s=CLOSED pgs=59 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.487 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.485+0000 7f01714296c0 1 -- 10.20.193.27:0/2697967832 >> 10.20.193.27:0/2697967832 conn(0x7f016c0ecb30 msgr2=0x7f016c10d890 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:30.487 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.485+0000 7f01714296c0 1 -- 10.20.193.27:0/2697967832 shutdown_connections 2026-02-13T02:22:30.487 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.485+0000 7f01714296c0 1 -- 10.20.193.27:0/2697967832 wait complete. 2026-02-13T02:22:30.490 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874247 got 279172874247 for osd.11 2026-02-13T02:22:30.490 DEBUG:teuthology.parallel:result is None 2026-02-13T02:22:30.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:30 trial028 ceph-mon[26701]: pgmap v190: 97 pgs: 97 active+clean; 629 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 6.2 KiB/s rd, 7.4 KiB/s wr, 8 op/s 2026-02-13T02:22:30.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:30 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3158728324' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2026-02-13T02:22:30.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:30 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2744320861' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2026-02-13T02:22:30.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:30 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3533716426' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2026-02-13T02:22:30.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:30 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3594651942' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2026-02-13T02:22:30.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:30 trial107 ceph-mon[27422]: pgmap v190: 97 pgs: 97 active+clean; 629 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 6.2 KiB/s rd, 7.4 KiB/s wr, 8 op/s 2026-02-13T02:22:30.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:30 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3158728324' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2026-02-13T02:22:30.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:30 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2744320861' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2026-02-13T02:22:30.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:30 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3533716426' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2026-02-13T02:22:30.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:30 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3594651942' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2026-02-13T02:22:30.517 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378644 got 81604378644 for osd.3 2026-02-13T02:22:30.517 DEBUG:teuthology.parallel:result is None 2026-02-13T02:22:30.519 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928233995 got 231928233995 for osd.9 2026-02-13T02:22:30.519 DEBUG:teuthology.parallel:result is None 2026-02-13T02:22:30.532 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691853 got 171798691853 for osd.7 2026-02-13T02:22:30.532 DEBUG:teuthology.parallel:result is None 2026-02-13T02:22:30.533 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462924 got 201863462924 for osd.8 2026-02-13T02:22:30.533 DEBUG:teuthology.parallel:result is None 2026-02-13T02:22:30.533 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2026-02-13T02:22:30.534 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph pg dump --format=json 2026-02-13T02:22:30.691 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:30.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.803+0000 7fa81e5a16c0 1 -- 10.20.193.27:0/1956210111 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa818115440 msgr2=0x7fa818175de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.803+0000 7fa81e5a16c0 1 --2- 10.20.193.27:0/1956210111 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa818115440 0x7fa818175de0 secure :-1 s=READY pgs=330 gs=1 cs=0 l=1 c_cookie=6199574463151024574 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa80c00aa70 tx=0x7fa80c02fde0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.803+0000 7fa81e5a16c0 1 -- 10.20.193.27:0/1956210111 shutdown_connections 2026-02-13T02:22:30.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.803+0000 7fa81e5a16c0 1 --2- 10.20.193.27:0/1956210111 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa818115440 0x7fa818175de0 unknown :-1 s=CLOSED pgs=330 gs=1 cs=0 l=1 c_cookie=6199574463151024574 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.803+0000 7fa81e5a16c0 1 --2- 10.20.193.27:0/1956210111 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa81810fc20 0x7fa818114c60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.803+0000 7fa81e5a16c0 1 --2- 10.20.193.27:0/1956210111 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa81810f230 0x7fa81810f600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.803+0000 7fa81e5a16c0 1 -- 10.20.193.27:0/1956210111 >> 10.20.193.27:0/1956210111 conn(0x7fa8180ecb30 msgr2=0x7fa81810cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:30.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.803+0000 7fa81e5a16c0 1 -- 10.20.193.27:0/1956210111 shutdown_connections 2026-02-13T02:22:30.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.803+0000 7fa81e5a16c0 1 -- 10.20.193.27:0/1956210111 wait complete. 2026-02-13T02:22:30.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.803+0000 7fa81e5a16c0 1 Processor -- start 2026-02-13T02:22:30.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.803+0000 7fa81e5a16c0 1 -- start start 2026-02-13T02:22:30.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.804+0000 7fa81e5a16c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa81810f230 0x7fa81811d200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.804+0000 7fa81e5a16c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa818115440 0x7fa81811d740 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.804+0000 7fa81e5a16c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa818122b00 0x7fa818122fa0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.804+0000 7fa81e5a16c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa818123510 con 0x7fa818115440 2026-02-13T02:22:30.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.804+0000 7fa81e5a16c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa818123680 con 0x7fa818122b00 2026-02-13T02:22:30.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.804+0000 7fa81e5a16c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa818123840 con 0x7fa81810f230 2026-02-13T02:22:30.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.804+0000 7fa8177fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa818115440 0x7fa81811d740 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.804+0000 7fa8177fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa818115440 0x7fa81811d740 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:54008/0 (socket says 10.20.193.27:54008) 2026-02-13T02:22:30.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.804+0000 7fa8177fe6c0 1 -- 10.20.193.27:0/378991308 learned_addr learned my addr 10.20.193.27:0/378991308 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:30.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.804+0000 7fa81cb136c0 1 --2- 10.20.193.27:0/378991308 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa818122b00 0x7fa818122fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.804+0000 7fa817fff6c0 1 --2- 10.20.193.27:0/378991308 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa81810f230 0x7fa81811d200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.804+0000 7fa8177fe6c0 1 -- 10.20.193.27:0/378991308 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa81810f230 msgr2=0x7fa81811d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.804+0000 7fa8177fe6c0 1 --2- 10.20.193.27:0/378991308 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa81810f230 0x7fa81811d200 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.809 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.804+0000 7fa8177fe6c0 1 -- 10.20.193.27:0/378991308 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa818122b00 msgr2=0x7fa818122fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.809 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.804+0000 7fa8177fe6c0 1 --2- 10.20.193.27:0/378991308 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa818122b00 0x7fa818122fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.809 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.804+0000 7fa8177fe6c0 1 -- 10.20.193.27:0/378991308 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa80c00a660 con 0x7fa818115440 2026-02-13T02:22:30.809 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.804+0000 7fa8177fe6c0 1 --2- 10.20.193.27:0/378991308 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa818115440 0x7fa81811d740 secure :-1 s=READY pgs=331 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa808005fd0 tx=0x7fa80800bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:30.809 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.804+0000 7fa8157fa6c0 1 -- 10.20.193.27:0/378991308 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa808004280 con 0x7fa818115440 2026-02-13T02:22:30.809 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.804+0000 7fa8157fa6c0 1 -- 10.20.193.27:0/378991308 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa8080043e0 con 0x7fa818115440 2026-02-13T02:22:30.809 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.804+0000 7fa81e5a16c0 1 -- 10.20.193.27:0/378991308 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa818163cf0 con 0x7fa818115440 2026-02-13T02:22:30.809 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.804+0000 7fa8157fa6c0 1 -- 10.20.193.27:0/378991308 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa80800ccd0 con 0x7fa818115440 2026-02-13T02:22:30.809 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.804+0000 7fa81e5a16c0 1 -- 10.20.193.27:0/378991308 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8181641d0 con 0x7fa818115440 2026-02-13T02:22:30.809 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.805+0000 7fa81e5a16c0 1 -- 10.20.193.27:0/378991308 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa818110500 con 0x7fa818115440 2026-02-13T02:22:30.810 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.806+0000 7fa8157fa6c0 1 -- 10.20.193.27:0/378991308 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7fa8080026b0 con 0x7fa818115440 2026-02-13T02:22:30.811 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.807+0000 7fa8157fa6c0 1 --2- 10.20.193.27:0/378991308 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa7ec0b5710 0x7fa7ec0d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:30.811 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.807+0000 7fa8157fa6c0 1 -- 10.20.193.27:0/378991308 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7fa8080e3a70 con 0x7fa818115440 2026-02-13T02:22:30.811 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.807+0000 7fa817fff6c0 1 --2- 10.20.193.27:0/378991308 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa7ec0b5710 0x7fa7ec0d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:30.811 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.807+0000 7fa8157fa6c0 1 -- 10.20.193.27:0/378991308 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fa80811c5e0 con 0x7fa818115440 2026-02-13T02:22:30.811 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.807+0000 7fa817fff6c0 1 --2- 10.20.193.27:0/378991308 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa7ec0b5710 0x7fa7ec0d5bc0 secure :-1 s=READY pgs=229 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa804005980 tx=0x7fa804005ee0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:30.898 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.896+0000 7fa81e5a16c0 1 -- 10.20.193.27:0/378991308 --> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa81810fc20 con 0x7fa7ec0b5710 2026-02-13T02:22:30.899 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.898+0000 7fa8157fa6c0 1 -- 10.20.193.27:0/378991308 <== mgr.14116 v2:10.20.193.27:6800/1837452956 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258877 (secure 0 0 0) 0x7fa81810fc20 con 0x7fa7ec0b5710 2026-02-13T02:22:30.902 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:22:30.904 INFO:teuthology.orchestra.run.trial027.stderr:dumped all 2026-02-13T02:22:30.906 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.900+0000 7fa81e5a16c0 1 -- 10.20.193.27:0/378991308 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa7ec0b5710 msgr2=0x7fa7ec0d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.906 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.900+0000 7fa81e5a16c0 1 --2- 10.20.193.27:0/378991308 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa7ec0b5710 0x7fa7ec0d5bc0 secure :-1 s=READY pgs=229 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa804005980 tx=0x7fa804005ee0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.907 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.900+0000 7fa81e5a16c0 1 -- 10.20.193.27:0/378991308 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa818115440 msgr2=0x7fa81811d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:30.907 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.900+0000 7fa81e5a16c0 1 --2- 10.20.193.27:0/378991308 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa818115440 0x7fa81811d740 secure :-1 s=READY pgs=331 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa808005fd0 tx=0x7fa80800bca0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.907 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.900+0000 7fa81e5a16c0 1 -- 10.20.193.27:0/378991308 shutdown_connections 2026-02-13T02:22:30.907 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.900+0000 7fa81e5a16c0 1 --2- 10.20.193.27:0/378991308 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa7ec0b5710 0x7fa7ec0d5bc0 unknown :-1 s=CLOSED pgs=229 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.907 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.900+0000 7fa81e5a16c0 1 --2- 10.20.193.27:0/378991308 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa818122b00 0x7fa818122fa0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.907 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.900+0000 7fa81e5a16c0 1 --2- 10.20.193.27:0/378991308 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa818115440 0x7fa81811d740 unknown :-1 s=CLOSED pgs=331 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.907 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.900+0000 7fa81e5a16c0 1 --2- 10.20.193.27:0/378991308 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa81810f230 0x7fa81811d200 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:30.907 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.900+0000 7fa81e5a16c0 1 -- 10.20.193.27:0/378991308 >> 10.20.193.27:0/378991308 conn(0x7fa8180ecb30 msgr2=0x7fa81810cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:30.907 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.900+0000 7fa81e5a16c0 1 -- 10.20.193.27:0/378991308 shutdown_connections 2026-02-13T02:22:30.907 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:30.900+0000 7fa81e5a16c0 1 -- 10.20.193.27:0/378991308 wait complete. 2026-02-13T02:22:30.940 INFO:teuthology.orchestra.run.trial027.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2026-02-13T02:22:28.958336+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644266,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":133,"num_read_kb":159,"num_write":256,"num_write_kb":3797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":8808038400,"kb_used":332756,"kb_used_data":9188,"kb_used_omap":330,"kb_used_meta":322997,"kb_avail":8807705644,"statfs":{"total":9019431321600,"available":9019090579456,"internally_reserved":0,"allocated":9408512,"data_stored":6296454,"data_compressed":114621,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":338584,"internal_metadata":330749288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":53898,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":27,"num_read_kb":68,"num_write":61,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.947296"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016636+0000","last_change":"2026-02-13T02:22:17.023288+0000","last_active":"2026-02-13T02:22:18.016636+0000","last_peered":"2026-02-13T02:22:18.016636+0000","last_clean":"2026-02-13T02:22:18.016636+0000","last_became_active":"2026-02-13T02:22:17.023073+0000","last_became_peered":"2026-02-13T02:22:17.023073+0000","last_unstale":"2026-02-13T02:22:18.016636+0000","last_undegraded":"2026-02-13T02:22:18.016636+0000","last_fullsized":"2026-02-13T02:22:18.016636+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:19:00.322079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047477+0000","last_change":"2026-02-13T02:22:17.018633+0000","last_active":"2026-02-13T02:22:18.047477+0000","last_peered":"2026-02-13T02:22:18.047477+0000","last_clean":"2026-02-13T02:22:18.047477+0000","last_became_active":"2026-02-13T02:22:17.018516+0000","last_became_peered":"2026-02-13T02:22:17.018516+0000","last_unstale":"2026-02-13T02:22:18.047477+0000","last_undegraded":"2026-02-13T02:22:18.047477+0000","last_fullsized":"2026-02-13T02:22:18.047477+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:33:08.892207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047500+0000","last_change":"2026-02-13T02:22:17.019231+0000","last_active":"2026-02-13T02:22:18.047500+0000","last_peered":"2026-02-13T02:22:18.047500+0000","last_clean":"2026-02-13T02:22:18.047500+0000","last_became_active":"2026-02-13T02:22:17.019039+0000","last_became_peered":"2026-02-13T02:22:17.019039+0000","last_unstale":"2026-02-13T02:22:18.047500+0000","last_undegraded":"2026-02-13T02:22:18.047500+0000","last_fullsized":"2026-02-13T02:22:18.047500+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:21:31.573485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016292+0000","last_change":"2026-02-13T02:22:17.021960+0000","last_active":"2026-02-13T02:22:18.016292+0000","last_peered":"2026-02-13T02:22:18.016292+0000","last_clean":"2026-02-13T02:22:18.016292+0000","last_became_active":"2026-02-13T02:22:17.021325+0000","last_became_peered":"2026-02-13T02:22:17.021325+0000","last_unstale":"2026-02-13T02:22:18.016292+0000","last_undegraded":"2026-02-13T02:22:18.016292+0000","last_fullsized":"2026-02-13T02:22:18.016292+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:28:01.069244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.024818+0000","last_change":"2026-02-13T02:22:17.022417+0000","last_active":"2026-02-13T02:22:18.024818+0000","last_peered":"2026-02-13T02:22:18.024818+0000","last_clean":"2026-02-13T02:22:18.024818+0000","last_became_active":"2026-02-13T02:22:17.022150+0000","last_became_peered":"2026-02-13T02:22:17.022150+0000","last_unstale":"2026-02-13T02:22:18.024818+0000","last_undegraded":"2026-02-13T02:22:18.024818+0000","last_fullsized":"2026-02-13T02:22:18.024818+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T12:26:21.518119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051355+0000","last_change":"2026-02-13T02:22:17.024197+0000","last_active":"2026-02-13T02:22:18.051355+0000","last_peered":"2026-02-13T02:22:18.051355+0000","last_clean":"2026-02-13T02:22:18.051355+0000","last_became_active":"2026-02-13T02:22:17.023378+0000","last_became_peered":"2026-02-13T02:22:17.023378+0000","last_unstale":"2026-02-13T02:22:18.051355+0000","last_undegraded":"2026-02-13T02:22:18.051355+0000","last_fullsized":"2026-02-13T02:22:18.051355+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:17:40.942731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051199+0000","last_change":"2026-02-13T02:22:17.021184+0000","last_active":"2026-02-13T02:22:18.051199+0000","last_peered":"2026-02-13T02:22:18.051199+0000","last_clean":"2026-02-13T02:22:18.051199+0000","last_became_active":"2026-02-13T02:22:17.020961+0000","last_became_peered":"2026-02-13T02:22:17.020961+0000","last_unstale":"2026-02-13T02:22:18.051199+0000","last_undegraded":"2026-02-13T02:22:18.051199+0000","last_fullsized":"2026-02-13T02:22:18.051199+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T10:38:19.326702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.020937+0000","last_change":"2026-02-13T02:22:17.021948+0000","last_active":"2026-02-13T02:22:18.020937+0000","last_peered":"2026-02-13T02:22:18.020937+0000","last_clean":"2026-02-13T02:22:18.020937+0000","last_became_active":"2026-02-13T02:22:17.021760+0000","last_became_peered":"2026-02-13T02:22:17.021760+0000","last_unstale":"2026-02-13T02:22:18.020937+0000","last_undegraded":"2026-02-13T02:22:18.020937+0000","last_fullsized":"2026-02-13T02:22:18.020937+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:24:58.456659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051727+0000","last_change":"2026-02-13T02:22:17.024036+0000","last_active":"2026-02-13T02:22:18.051727+0000","last_peered":"2026-02-13T02:22:18.051727+0000","last_clean":"2026-02-13T02:22:18.051727+0000","last_became_active":"2026-02-13T02:22:17.023805+0000","last_became_peered":"2026-02-13T02:22:17.023805+0000","last_unstale":"2026-02-13T02:22:18.051727+0000","last_undegraded":"2026-02-13T02:22:18.051727+0000","last_fullsized":"2026-02-13T02:22:18.051727+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T02:44:29.738084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051866+0000","last_change":"2026-02-13T02:22:17.024557+0000","last_active":"2026-02-13T02:22:18.051866+0000","last_peered":"2026-02-13T02:22:18.051866+0000","last_clean":"2026-02-13T02:22:18.051866+0000","last_became_active":"2026-02-13T02:22:17.024270+0000","last_became_peered":"2026-02-13T02:22:17.024270+0000","last_unstale":"2026-02-13T02:22:18.051866+0000","last_undegraded":"2026-02-13T02:22:18.051866+0000","last_fullsized":"2026-02-13T02:22:18.051866+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:36:05.815116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051729+0000","last_change":"2026-02-13T02:22:17.023942+0000","last_active":"2026-02-13T02:22:18.051729+0000","last_peered":"2026-02-13T02:22:18.051729+0000","last_clean":"2026-02-13T02:22:18.051729+0000","last_became_active":"2026-02-13T02:22:17.023475+0000","last_became_peered":"2026-02-13T02:22:17.023475+0000","last_unstale":"2026-02-13T02:22:18.051729+0000","last_undegraded":"2026-02-13T02:22:18.051729+0000","last_fullsized":"2026-02-13T02:22:18.051729+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T12:17:44.718186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016376+0000","last_change":"2026-02-13T02:22:17.022471+0000","last_active":"2026-02-13T02:22:18.016376+0000","last_peered":"2026-02-13T02:22:18.016376+0000","last_clean":"2026-02-13T02:22:18.016376+0000","last_became_active":"2026-02-13T02:22:17.021894+0000","last_became_peered":"2026-02-13T02:22:17.021894+0000","last_unstale":"2026-02-13T02:22:18.016376+0000","last_undegraded":"2026-02-13T02:22:18.016376+0000","last_fullsized":"2026-02-13T02:22:18.016376+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:27:33.483596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047429+0000","last_change":"2026-02-13T02:22:17.017830+0000","last_active":"2026-02-13T02:22:18.047429+0000","last_peered":"2026-02-13T02:22:18.047429+0000","last_clean":"2026-02-13T02:22:18.047429+0000","last_became_active":"2026-02-13T02:22:17.017232+0000","last_became_peered":"2026-02-13T02:22:17.017232+0000","last_unstale":"2026-02-13T02:22:18.047429+0000","last_undegraded":"2026-02-13T02:22:18.047429+0000","last_fullsized":"2026-02-13T02:22:18.047429+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:40:47.725661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047355+0000","last_change":"2026-02-13T02:22:17.021764+0000","last_active":"2026-02-13T02:22:18.047355+0000","last_peered":"2026-02-13T02:22:18.047355+0000","last_clean":"2026-02-13T02:22:18.047355+0000","last_became_active":"2026-02-13T02:22:17.021609+0000","last_became_peered":"2026-02-13T02:22:17.021609+0000","last_unstale":"2026-02-13T02:22:18.047355+0000","last_undegraded":"2026-02-13T02:22:18.047355+0000","last_fullsized":"2026-02-13T02:22:18.047355+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T12:04:31.168034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047332+0000","last_change":"2026-02-13T02:22:17.019584+0000","last_active":"2026-02-13T02:22:18.047332+0000","last_peered":"2026-02-13T02:22:18.047332+0000","last_clean":"2026-02-13T02:22:18.047332+0000","last_became_active":"2026-02-13T02:22:17.019388+0000","last_became_peered":"2026-02-13T02:22:17.019388+0000","last_unstale":"2026-02-13T02:22:18.047332+0000","last_undegraded":"2026-02-13T02:22:18.047332+0000","last_fullsized":"2026-02-13T02:22:18.047332+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:33:16.084048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.050998+0000","last_change":"2026-02-13T02:22:17.023950+0000","last_active":"2026-02-13T02:22:18.050998+0000","last_peered":"2026-02-13T02:22:18.050998+0000","last_clean":"2026-02-13T02:22:18.050998+0000","last_became_active":"2026-02-13T02:22:17.023029+0000","last_became_peered":"2026-02-13T02:22:17.023029+0000","last_unstale":"2026-02-13T02:22:18.050998+0000","last_undegraded":"2026-02-13T02:22:18.050998+0000","last_fullsized":"2026-02-13T02:22:18.050998+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:14:25.119381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016670+0000","last_change":"2026-02-13T02:22:17.024675+0000","last_active":"2026-02-13T02:22:18.016670+0000","last_peered":"2026-02-13T02:22:18.016670+0000","last_clean":"2026-02-13T02:22:18.016670+0000","last_became_active":"2026-02-13T02:22:17.024460+0000","last_became_peered":"2026-02-13T02:22:17.024460+0000","last_unstale":"2026-02-13T02:22:18.016670+0000","last_undegraded":"2026-02-13T02:22:18.016670+0000","last_fullsized":"2026-02-13T02:22:18.016670+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T04:41:20.995149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.012018+0000","last_change":"2026-02-13T02:22:17.019261+0000","last_active":"2026-02-13T02:22:18.012018+0000","last_peered":"2026-02-13T02:22:18.012018+0000","last_clean":"2026-02-13T02:22:18.012018+0000","last_became_active":"2026-02-13T02:22:17.019084+0000","last_became_peered":"2026-02-13T02:22:17.019084+0000","last_unstale":"2026-02-13T02:22:18.012018+0000","last_undegraded":"2026-02-13T02:22:18.012018+0000","last_fullsized":"2026-02-13T02:22:18.012018+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:26:06.852915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016186+0000","last_change":"2026-02-13T02:22:17.021336+0000","last_active":"2026-02-13T02:22:18.016186+0000","last_peered":"2026-02-13T02:22:18.016186+0000","last_clean":"2026-02-13T02:22:18.016186+0000","last_became_active":"2026-02-13T02:22:17.020885+0000","last_became_peered":"2026-02-13T02:22:17.020885+0000","last_unstale":"2026-02-13T02:22:18.016186+0000","last_undegraded":"2026-02-13T02:22:18.016186+0000","last_fullsized":"2026-02-13T02:22:18.016186+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T14:05:29.982317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016382+0000","last_change":"2026-02-13T02:22:17.022900+0000","last_active":"2026-02-13T02:22:18.016382+0000","last_peered":"2026-02-13T02:22:18.016382+0000","last_clean":"2026-02-13T02:22:18.016382+0000","last_became_active":"2026-02-13T02:22:17.022753+0000","last_became_peered":"2026-02-13T02:22:17.022753+0000","last_unstale":"2026-02-13T02:22:18.016382+0000","last_undegraded":"2026-02-13T02:22:18.016382+0000","last_fullsized":"2026-02-13T02:22:18.016382+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:35:07.485606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047442+0000","last_change":"2026-02-13T02:22:17.018325+0000","last_active":"2026-02-13T02:22:18.047442+0000","last_peered":"2026-02-13T02:22:18.047442+0000","last_clean":"2026-02-13T02:22:18.047442+0000","last_became_active":"2026-02-13T02:22:17.018167+0000","last_became_peered":"2026-02-13T02:22:17.018167+0000","last_unstale":"2026-02-13T02:22:18.047442+0000","last_undegraded":"2026-02-13T02:22:18.047442+0000","last_fullsized":"2026-02-13T02:22:18.047442+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T14:09:03.876188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051256+0000","last_change":"2026-02-13T02:22:17.022546+0000","last_active":"2026-02-13T02:22:18.051256+0000","last_peered":"2026-02-13T02:22:18.051256+0000","last_clean":"2026-02-13T02:22:18.051256+0000","last_became_active":"2026-02-13T02:22:17.022064+0000","last_became_peered":"2026-02-13T02:22:17.022064+0000","last_unstale":"2026-02-13T02:22:18.051256+0000","last_undegraded":"2026-02-13T02:22:18.051256+0000","last_fullsized":"2026-02-13T02:22:18.051256+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:55:49.327407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.020899+0000","last_change":"2026-02-13T02:22:17.018592+0000","last_active":"2026-02-13T02:22:18.020899+0000","last_peered":"2026-02-13T02:22:18.020899+0000","last_clean":"2026-02-13T02:22:18.020899+0000","last_became_active":"2026-02-13T02:22:17.018368+0000","last_became_peered":"2026-02-13T02:22:17.018368+0000","last_unstale":"2026-02-13T02:22:18.020899+0000","last_undegraded":"2026-02-13T02:22:18.020899+0000","last_fullsized":"2026-02-13T02:22:18.020899+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:15:11.963709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051251+0000","last_change":"2026-02-13T02:22:17.023462+0000","last_active":"2026-02-13T02:22:18.051251+0000","last_peered":"2026-02-13T02:22:18.051251+0000","last_clean":"2026-02-13T02:22:18.051251+0000","last_became_active":"2026-02-13T02:22:17.023327+0000","last_became_peered":"2026-02-13T02:22:17.023327+0000","last_unstale":"2026-02-13T02:22:18.051251+0000","last_undegraded":"2026-02-13T02:22:18.051251+0000","last_fullsized":"2026-02-13T02:22:18.051251+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:42:27.909493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.021321+0000","last_change":"2026-02-13T02:22:17.021211+0000","last_active":"2026-02-13T02:22:18.021321+0000","last_peered":"2026-02-13T02:22:18.021321+0000","last_clean":"2026-02-13T02:22:18.021321+0000","last_became_active":"2026-02-13T02:22:17.021002+0000","last_became_peered":"2026-02-13T02:22:17.021002+0000","last_unstale":"2026-02-13T02:22:18.021321+0000","last_undegraded":"2026-02-13T02:22:18.021321+0000","last_fullsized":"2026-02-13T02:22:18.021321+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:23:00.242280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016321+0000","last_change":"2026-02-13T02:22:17.023595+0000","last_active":"2026-02-13T02:22:18.016321+0000","last_peered":"2026-02-13T02:22:18.016321+0000","last_clean":"2026-02-13T02:22:18.016321+0000","last_became_active":"2026-02-13T02:22:17.023480+0000","last_became_peered":"2026-02-13T02:22:17.023480+0000","last_unstale":"2026-02-13T02:22:18.016321+0000","last_undegraded":"2026-02-13T02:22:18.016321+0000","last_fullsized":"2026-02-13T02:22:18.016321+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:13:14.505114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016555+0000","last_change":"2026-02-13T02:22:17.021599+0000","last_active":"2026-02-13T02:22:18.016555+0000","last_peered":"2026-02-13T02:22:18.016555+0000","last_clean":"2026-02-13T02:22:18.016555+0000","last_became_active":"2026-02-13T02:22:17.021373+0000","last_became_peered":"2026-02-13T02:22:17.021373+0000","last_unstale":"2026-02-13T02:22:18.016555+0000","last_undegraded":"2026-02-13T02:22:18.016555+0000","last_fullsized":"2026-02-13T02:22:18.016555+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:35:13.390476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047383+0000","last_change":"2026-02-13T02:22:17.017507+0000","last_active":"2026-02-13T02:22:18.047383+0000","last_peered":"2026-02-13T02:22:18.047383+0000","last_clean":"2026-02-13T02:22:18.047383+0000","last_became_active":"2026-02-13T02:22:17.017211+0000","last_became_peered":"2026-02-13T02:22:17.017211+0000","last_unstale":"2026-02-13T02:22:18.047383+0000","last_undegraded":"2026-02-13T02:22:18.047383+0000","last_fullsized":"2026-02-13T02:22:18.047383+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:19:19.531222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047561+0000","last_change":"2026-02-13T02:22:17.019486+0000","last_active":"2026-02-13T02:22:18.047561+0000","last_peered":"2026-02-13T02:22:18.047561+0000","last_clean":"2026-02-13T02:22:18.047561+0000","last_became_active":"2026-02-13T02:22:17.019288+0000","last_became_peered":"2026-02-13T02:22:17.019288+0000","last_unstale":"2026-02-13T02:22:18.047561+0000","last_undegraded":"2026-02-13T02:22:18.047561+0000","last_fullsized":"2026-02-13T02:22:18.047561+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:51:14.055654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.015982+0000","last_change":"2026-02-13T02:22:17.022029+0000","last_active":"2026-02-13T02:22:18.015982+0000","last_peered":"2026-02-13T02:22:18.015982+0000","last_clean":"2026-02-13T02:22:18.015982+0000","last_became_active":"2026-02-13T02:22:17.021771+0000","last_became_peered":"2026-02-13T02:22:17.021771+0000","last_unstale":"2026-02-13T02:22:18.015982+0000","last_undegraded":"2026-02-13T02:22:18.015982+0000","last_fullsized":"2026-02-13T02:22:18.015982+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:15:51.936117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016131+0000","last_change":"2026-02-13T02:22:17.021554+0000","last_active":"2026-02-13T02:22:18.016131+0000","last_peered":"2026-02-13T02:22:18.016131+0000","last_clean":"2026-02-13T02:22:18.016131+0000","last_became_active":"2026-02-13T02:22:17.021248+0000","last_became_peered":"2026-02-13T02:22:17.021248+0000","last_unstale":"2026-02-13T02:22:18.016131+0000","last_undegraded":"2026-02-13T02:22:18.016131+0000","last_fullsized":"2026-02-13T02:22:18.016131+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T12:27:26.132715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016224+0000","last_change":"2026-02-13T02:22:17.022426+0000","last_active":"2026-02-13T02:22:18.016224+0000","last_peered":"2026-02-13T02:22:18.016224+0000","last_clean":"2026-02-13T02:22:18.016224+0000","last_became_active":"2026-02-13T02:22:17.022285+0000","last_became_peered":"2026-02-13T02:22:17.022285+0000","last_unstale":"2026-02-13T02:22:18.016224+0000","last_undegraded":"2026-02-13T02:22:18.016224+0000","last_fullsized":"2026-02-13T02:22:18.016224+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:25:58.151252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.024814+0000","last_change":"2026-02-13T02:22:17.022749+0000","last_active":"2026-02-13T02:22:18.024814+0000","last_peered":"2026-02-13T02:22:18.024814+0000","last_clean":"2026-02-13T02:22:18.024814+0000","last_became_active":"2026-02-13T02:22:17.022397+0000","last_became_peered":"2026-02-13T02:22:17.022397+0000","last_unstale":"2026-02-13T02:22:18.024814+0000","last_undegraded":"2026-02-13T02:22:18.024814+0000","last_fullsized":"2026-02-13T02:22:18.024814+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:28:44.768608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"69'1","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051275+0000","last_change":"2026-02-13T02:22:16.014721+0000","last_active":"2026-02-13T02:22:18.051275+0000","last_peered":"2026-02-13T02:22:18.051275+0000","last_clean":"2026-02-13T02:22:18.051275+0000","last_became_active":"2026-02-13T02:22:16.014587+0000","last_became_peered":"2026-02-13T02:22:16.014587+0000","last_unstale":"2026-02-13T02:22:18.051275+0000","last_undegraded":"2026-02-13T02:22:18.051275+0000","last_fullsized":"2026-02-13T02:22:18.051275+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:56:58.003775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051261+0000","last_change":"2026-02-13T02:22:17.022514+0000","last_active":"2026-02-13T02:22:18.051261+0000","last_peered":"2026-02-13T02:22:18.051261+0000","last_clean":"2026-02-13T02:22:18.051261+0000","last_became_active":"2026-02-13T02:22:17.022273+0000","last_became_peered":"2026-02-13T02:22:17.022273+0000","last_unstale":"2026-02-13T02:22:18.051261+0000","last_undegraded":"2026-02-13T02:22:18.051261+0000","last_fullsized":"2026-02-13T02:22:18.051261+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:33:27.805740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"69'1","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051311+0000","last_change":"2026-02-13T02:22:16.012690+0000","last_active":"2026-02-13T02:22:18.051311+0000","last_peered":"2026-02-13T02:22:18.051311+0000","last_clean":"2026-02-13T02:22:18.051311+0000","last_became_active":"2026-02-13T02:22:16.011935+0000","last_became_peered":"2026-02-13T02:22:16.011935+0000","last_unstale":"2026-02-13T02:22:18.051311+0000","last_undegraded":"2026-02-13T02:22:18.051311+0000","last_fullsized":"2026-02-13T02:22:18.051311+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T06:55:46.440839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047121+0000","last_change":"2026-02-13T02:22:16.007738+0000","last_active":"2026-02-13T02:22:18.047121+0000","last_peered":"2026-02-13T02:22:18.047121+0000","last_clean":"2026-02-13T02:22:18.047121+0000","last_became_active":"2026-02-13T02:22:16.007424+0000","last_became_peered":"2026-02-13T02:22:16.007424+0000","last_unstale":"2026-02-13T02:22:18.047121+0000","last_undegraded":"2026-02-13T02:22:18.047121+0000","last_fullsized":"2026-02-13T02:22:18.047121+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:10:41.888980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.020958+0000","last_change":"2026-02-13T02:22:17.017332+0000","last_active":"2026-02-13T02:22:18.020958+0000","last_peered":"2026-02-13T02:22:18.020958+0000","last_clean":"2026-02-13T02:22:18.020958+0000","last_became_active":"2026-02-13T02:22:17.017080+0000","last_became_peered":"2026-02-13T02:22:17.017080+0000","last_unstale":"2026-02-13T02:22:18.020958+0000","last_undegraded":"2026-02-13T02:22:18.020958+0000","last_fullsized":"2026-02-13T02:22:18.020958+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:10:16.349029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051313+0000","last_change":"2026-02-13T02:22:16.014204+0000","last_active":"2026-02-13T02:22:18.051313+0000","last_peered":"2026-02-13T02:22:18.051313+0000","last_clean":"2026-02-13T02:22:18.051313+0000","last_became_active":"2026-02-13T02:22:16.014114+0000","last_became_peered":"2026-02-13T02:22:16.014114+0000","last_unstale":"2026-02-13T02:22:18.051313+0000","last_undegraded":"2026-02-13T02:22:18.051313+0000","last_fullsized":"2026-02-13T02:22:18.051313+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:39:42.557823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.011965+0000","last_change":"2026-02-13T02:22:17.018644+0000","last_active":"2026-02-13T02:22:18.011965+0000","last_peered":"2026-02-13T02:22:18.011965+0000","last_clean":"2026-02-13T02:22:18.011965+0000","last_became_active":"2026-02-13T02:22:17.018511+0000","last_became_peered":"2026-02-13T02:22:17.018511+0000","last_unstale":"2026-02-13T02:22:18.011965+0000","last_undegraded":"2026-02-13T02:22:18.011965+0000","last_fullsized":"2026-02-13T02:22:18.011965+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:18:39.510344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051718+0000","last_change":"2026-02-13T02:22:16.013123+0000","last_active":"2026-02-13T02:22:18.051718+0000","last_peered":"2026-02-13T02:22:18.051718+0000","last_clean":"2026-02-13T02:22:18.051718+0000","last_became_active":"2026-02-13T02:22:16.012986+0000","last_became_peered":"2026-02-13T02:22:16.012986+0000","last_unstale":"2026-02-13T02:22:18.051718+0000","last_undegraded":"2026-02-13T02:22:18.051718+0000","last_fullsized":"2026-02-13T02:22:18.051718+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T10:09:29.447645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051655+0000","last_change":"2026-02-13T02:22:17.024718+0000","last_active":"2026-02-13T02:22:18.051655+0000","last_peered":"2026-02-13T02:22:18.051655+0000","last_clean":"2026-02-13T02:22:18.051655+0000","last_became_active":"2026-02-13T02:22:17.024333+0000","last_became_peered":"2026-02-13T02:22:17.024333+0000","last_unstale":"2026-02-13T02:22:18.051655+0000","last_undegraded":"2026-02-13T02:22:18.051655+0000","last_fullsized":"2026-02-13T02:22:18.051655+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:44:11.055118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051693+0000","last_change":"2026-02-13T02:22:16.015080+0000","last_active":"2026-02-13T02:22:18.051693+0000","last_peered":"2026-02-13T02:22:18.051693+0000","last_clean":"2026-02-13T02:22:18.051693+0000","last_became_active":"2026-02-13T02:22:16.014694+0000","last_became_peered":"2026-02-13T02:22:16.014694+0000","last_unstale":"2026-02-13T02:22:18.051693+0000","last_undegraded":"2026-02-13T02:22:18.051693+0000","last_fullsized":"2026-02-13T02:22:18.051693+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T06:02:28.163616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047603+0000","last_change":"2026-02-13T02:22:17.019516+0000","last_active":"2026-02-13T02:22:18.047603+0000","last_peered":"2026-02-13T02:22:18.047603+0000","last_clean":"2026-02-13T02:22:18.047603+0000","last_became_active":"2026-02-13T02:22:17.019234+0000","last_became_peered":"2026-02-13T02:22:17.019234+0000","last_unstale":"2026-02-13T02:22:18.047603+0000","last_undegraded":"2026-02-13T02:22:18.047603+0000","last_fullsized":"2026-02-13T02:22:18.047603+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T10:25:31.467398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"69'3","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051378+0000","last_change":"2026-02-13T02:22:16.013733+0000","last_active":"2026-02-13T02:22:18.051378+0000","last_peered":"2026-02-13T02:22:18.051378+0000","last_clean":"2026-02-13T02:22:18.051378+0000","last_became_active":"2026-02-13T02:22:16.013472+0000","last_became_peered":"2026-02-13T02:22:16.013472+0000","last_unstale":"2026-02-13T02:22:18.051378+0000","last_undegraded":"2026-02-13T02:22:18.051378+0000","last_fullsized":"2026-02-13T02:22:18.051378+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:59:36.928070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051362+0000","last_change":"2026-02-13T02:22:17.023907+0000","last_active":"2026-02-13T02:22:18.051362+0000","last_peered":"2026-02-13T02:22:18.051362+0000","last_clean":"2026-02-13T02:22:18.051362+0000","last_became_active":"2026-02-13T02:22:17.022420+0000","last_became_peered":"2026-02-13T02:22:17.022420+0000","last_unstale":"2026-02-13T02:22:18.051362+0000","last_undegraded":"2026-02-13T02:22:18.051362+0000","last_fullsized":"2026-02-13T02:22:18.051362+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:29:25.440291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"70'20","reported_seq":57,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:27.041983+0000","last_change":"2026-02-13T02:22:16.013888+0000","last_active":"2026-02-13T02:22:27.041983+0000","last_peered":"2026-02-13T02:22:27.041983+0000","last_clean":"2026-02-13T02:22:27.041983+0000","last_became_active":"2026-02-13T02:22:16.013652+0000","last_became_peered":"2026-02-13T02:22:16.013652+0000","last_unstale":"2026-02-13T02:22:27.041983+0000","last_undegraded":"2026-02-13T02:22:27.041983+0000","last_fullsized":"2026-02-13T02:22:27.041983+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:25:28.136946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52844,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":62,"num_write":20,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.025043+0000","last_change":"2026-02-13T02:22:17.023547+0000","last_active":"2026-02-13T02:22:18.025043+0000","last_peered":"2026-02-13T02:22:18.025043+0000","last_clean":"2026-02-13T02:22:18.025043+0000","last_became_active":"2026-02-13T02:22:17.023410+0000","last_became_peered":"2026-02-13T02:22:17.023410+0000","last_unstale":"2026-02-13T02:22:18.025043+0000","last_undegraded":"2026-02-13T02:22:18.025043+0000","last_fullsized":"2026-02-13T02:22:18.025043+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T04:35:03.522242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"69'2","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.507624+0000","last_change":"2026-02-13T02:22:16.014701+0000","last_active":"2026-02-13T02:22:18.507624+0000","last_peered":"2026-02-13T02:22:18.507624+0000","last_clean":"2026-02-13T02:22:18.507624+0000","last_became_active":"2026-02-13T02:22:16.014476+0000","last_became_peered":"2026-02-13T02:22:16.014476+0000","last_unstale":"2026-02-13T02:22:18.507624+0000","last_undegraded":"2026-02-13T02:22:18.507624+0000","last_fullsized":"2026-02-13T02:22:18.507624+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:25:47.865783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047309+0000","last_change":"2026-02-13T02:22:17.017924+0000","last_active":"2026-02-13T02:22:18.047309+0000","last_peered":"2026-02-13T02:22:18.047309+0000","last_clean":"2026-02-13T02:22:18.047309+0000","last_became_active":"2026-02-13T02:22:17.017666+0000","last_became_peered":"2026-02-13T02:22:17.017666+0000","last_unstale":"2026-02-13T02:22:18.047309+0000","last_undegraded":"2026-02-13T02:22:18.047309+0000","last_fullsized":"2026-02-13T02:22:18.047309+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:38:38.063114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051219+0000","last_change":"2026-02-13T02:22:16.014437+0000","last_active":"2026-02-13T02:22:18.051219+0000","last_peered":"2026-02-13T02:22:18.051219+0000","last_clean":"2026-02-13T02:22:18.051219+0000","last_became_active":"2026-02-13T02:22:16.014293+0000","last_became_peered":"2026-02-13T02:22:16.014293+0000","last_unstale":"2026-02-13T02:22:18.051219+0000","last_undegraded":"2026-02-13T02:22:18.051219+0000","last_fullsized":"2026-02-13T02:22:18.051219+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:08:58.340775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051558+0000","last_change":"2026-02-13T02:22:17.023621+0000","last_active":"2026-02-13T02:22:18.051558+0000","last_peered":"2026-02-13T02:22:18.051558+0000","last_clean":"2026-02-13T02:22:18.051558+0000","last_became_active":"2026-02-13T02:22:17.023343+0000","last_became_peered":"2026-02-13T02:22:17.023343+0000","last_unstale":"2026-02-13T02:22:18.051558+0000","last_undegraded":"2026-02-13T02:22:18.051558+0000","last_fullsized":"2026-02-13T02:22:18.051558+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T06:36:08.489377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.015962+0000","last_change":"2026-02-13T02:22:16.013825+0000","last_active":"2026-02-13T02:22:18.015962+0000","last_peered":"2026-02-13T02:22:18.015962+0000","last_clean":"2026-02-13T02:22:18.015962+0000","last_became_active":"2026-02-13T02:22:16.013704+0000","last_became_peered":"2026-02-13T02:22:16.013704+0000","last_unstale":"2026-02-13T02:22:18.015962+0000","last_undegraded":"2026-02-13T02:22:18.015962+0000","last_fullsized":"2026-02-13T02:22:18.015962+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T02:40:53.106423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051148+0000","last_change":"2026-02-13T02:22:17.022293+0000","last_active":"2026-02-13T02:22:18.051148+0000","last_peered":"2026-02-13T02:22:18.051148+0000","last_clean":"2026-02-13T02:22:18.051148+0000","last_became_active":"2026-02-13T02:22:17.022130+0000","last_became_peered":"2026-02-13T02:22:17.022130+0000","last_unstale":"2026-02-13T02:22:18.051148+0000","last_undegraded":"2026-02-13T02:22:18.051148+0000","last_fullsized":"2026-02-13T02:22:18.051148+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:33:03.123178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"69'8","reported_seq":24,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.024641+0000","last_change":"2026-02-13T02:22:16.013612+0000","last_active":"2026-02-13T02:22:18.024641+0000","last_peered":"2026-02-13T02:22:18.024641+0000","last_clean":"2026-02-13T02:22:18.024641+0000","last_became_active":"2026-02-13T02:22:16.013444+0000","last_became_peered":"2026-02-13T02:22:16.013444+0000","last_unstale":"2026-02-13T02:22:18.024641+0000","last_undegraded":"2026-02-13T02:22:18.024641+0000","last_fullsized":"2026-02-13T02:22:18.024641+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T04:59:32.263657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051261+0000","last_change":"2026-02-13T02:22:17.024692+0000","last_active":"2026-02-13T02:22:18.051261+0000","last_peered":"2026-02-13T02:22:18.051261+0000","last_clean":"2026-02-13T02:22:18.051261+0000","last_became_active":"2026-02-13T02:22:17.024543+0000","last_became_peered":"2026-02-13T02:22:17.024543+0000","last_unstale":"2026-02-13T02:22:18.051261+0000","last_undegraded":"2026-02-13T02:22:18.051261+0000","last_fullsized":"2026-02-13T02:22:18.051261+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:04:25.726793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"69'1","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.025073+0000","last_change":"2026-02-13T02:22:16.011678+0000","last_active":"2026-02-13T02:22:18.025073+0000","last_peered":"2026-02-13T02:22:18.025073+0000","last_clean":"2026-02-13T02:22:18.025073+0000","last_became_active":"2026-02-13T02:22:16.010914+0000","last_became_peered":"2026-02-13T02:22:16.010914+0000","last_unstale":"2026-02-13T02:22:18.025073+0000","last_undegraded":"2026-02-13T02:22:18.025073+0000","last_fullsized":"2026-02-13T02:22:18.025073+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:44:29.583923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016496+0000","last_change":"2026-02-13T02:22:17.023312+0000","last_active":"2026-02-13T02:22:18.016496+0000","last_peered":"2026-02-13T02:22:18.016496+0000","last_clean":"2026-02-13T02:22:18.016496+0000","last_became_active":"2026-02-13T02:22:17.023084+0000","last_became_peered":"2026-02-13T02:22:17.023084+0000","last_unstale":"2026-02-13T02:22:18.016496+0000","last_undegraded":"2026-02-13T02:22:18.016496+0000","last_fullsized":"2026-02-13T02:22:18.016496+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T10:27:31.848680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"69'2","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047389+0000","last_change":"2026-02-13T02:22:16.010281+0000","last_active":"2026-02-13T02:22:18.047389+0000","last_peered":"2026-02-13T02:22:18.047389+0000","last_clean":"2026-02-13T02:22:18.047389+0000","last_became_active":"2026-02-13T02:22:16.010075+0000","last_became_peered":"2026-02-13T02:22:16.010075+0000","last_unstale":"2026-02-13T02:22:18.047389+0000","last_undegraded":"2026-02-13T02:22:18.047389+0000","last_fullsized":"2026-02-13T02:22:18.047389+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:25:52.223102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051820+0000","last_change":"2026-02-13T02:22:17.023958+0000","last_active":"2026-02-13T02:22:18.051820+0000","last_peered":"2026-02-13T02:22:18.051820+0000","last_clean":"2026-02-13T02:22:18.051820+0000","last_became_active":"2026-02-13T02:22:17.023529+0000","last_became_peered":"2026-02-13T02:22:17.023529+0000","last_unstale":"2026-02-13T02:22:18.051820+0000","last_undegraded":"2026-02-13T02:22:18.051820+0000","last_fullsized":"2026-02-13T02:22:18.051820+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T12:45:26.937375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"69'1","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:19.304376+0000","last_change":"2026-02-13T02:22:16.013663+0000","last_active":"2026-02-13T02:22:19.304376+0000","last_peered":"2026-02-13T02:22:19.304376+0000","last_clean":"2026-02-13T02:22:19.304376+0000","last_became_active":"2026-02-13T02:22:16.013445+0000","last_became_peered":"2026-02-13T02:22:16.013445+0000","last_unstale":"2026-02-13T02:22:19.304376+0000","last_undegraded":"2026-02-13T02:22:19.304376+0000","last_fullsized":"2026-02-13T02:22:19.304376+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:02:58.149380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"70'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:19.481305+0000","last_change":"2026-02-13T02:22:17.021199+0000","last_active":"2026-02-13T02:22:19.481305+0000","last_peered":"2026-02-13T02:22:19.481305+0000","last_clean":"2026-02-13T02:22:19.481305+0000","last_became_active":"2026-02-13T02:22:17.021008+0000","last_became_peered":"2026-02-13T02:22:17.021008+0000","last_unstale":"2026-02-13T02:22:19.481305+0000","last_undegraded":"2026-02-13T02:22:19.481305+0000","last_fullsized":"2026-02-13T02:22:19.481305+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:21:08.010400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016275+0000","last_change":"2026-02-13T02:22:16.017721+0000","last_active":"2026-02-13T02:22:18.016275+0000","last_peered":"2026-02-13T02:22:18.016275+0000","last_clean":"2026-02-13T02:22:18.016275+0000","last_became_active":"2026-02-13T02:22:16.017552+0000","last_became_peered":"2026-02-13T02:22:16.017552+0000","last_unstale":"2026-02-13T02:22:18.016275+0000","last_undegraded":"2026-02-13T02:22:18.016275+0000","last_fullsized":"2026-02-13T02:22:18.016275+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:14:18.034709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051822+0000","last_change":"2026-02-13T02:22:17.025362+0000","last_active":"2026-02-13T02:22:18.051822+0000","last_peered":"2026-02-13T02:22:18.051822+0000","last_clean":"2026-02-13T02:22:18.051822+0000","last_became_active":"2026-02-13T02:22:17.024942+0000","last_became_peered":"2026-02-13T02:22:17.024942+0000","last_unstale":"2026-02-13T02:22:18.051822+0000","last_undegraded":"2026-02-13T02:22:18.051822+0000","last_fullsized":"2026-02-13T02:22:18.051822+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:09:37.454791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'152","reported_seq":339,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051251+0000","last_change":"2026-02-13T02:21:55.384973+0000","last_active":"2026-02-13T02:22:18.051251+0000","last_peered":"2026-02-13T02:22:18.051251+0000","last_clean":"2026-02-13T02:22:18.051251+0000","last_became_active":"2026-02-13T02:21:55.384610+0000","last_became_peered":"2026-02-13T02:21:55.384610+0000","last_unstale":"2026-02-13T02:22:18.051251+0000","last_undegraded":"2026-02-13T02:22:18.051251+0000","last_fullsized":"2026-02-13T02:22:18.051251+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:20:55.621197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:20:55.621197+0000","last_clean_scrub_stamp":"2026-02-13T02:20:55.621197+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:45:14.548172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":189,"num_write_kb":3716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016150+0000","last_change":"2026-02-13T02:22:16.012172+0000","last_active":"2026-02-13T02:22:18.016150+0000","last_peered":"2026-02-13T02:22:18.016150+0000","last_clean":"2026-02-13T02:22:18.016150+0000","last_became_active":"2026-02-13T02:22:16.011646+0000","last_became_peered":"2026-02-13T02:22:16.011646+0000","last_unstale":"2026-02-13T02:22:18.016150+0000","last_undegraded":"2026-02-13T02:22:18.016150+0000","last_fullsized":"2026-02-13T02:22:18.016150+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:05:01.007415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.021079+0000","last_change":"2026-02-13T02:22:17.020741+0000","last_active":"2026-02-13T02:22:18.021079+0000","last_peered":"2026-02-13T02:22:18.021079+0000","last_clean":"2026-02-13T02:22:18.021079+0000","last_became_active":"2026-02-13T02:22:17.020574+0000","last_became_peered":"2026-02-13T02:22:17.020574+0000","last_unstale":"2026-02-13T02:22:18.021079+0000","last_undegraded":"2026-02-13T02:22:18.021079+0000","last_fullsized":"2026-02-13T02:22:18.021079+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T06:28:11.279493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051246+0000","last_change":"2026-02-13T02:22:16.014565+0000","last_active":"2026-02-13T02:22:18.051246+0000","last_peered":"2026-02-13T02:22:18.051246+0000","last_clean":"2026-02-13T02:22:18.051246+0000","last_became_active":"2026-02-13T02:22:16.013815+0000","last_became_peered":"2026-02-13T02:22:16.013815+0000","last_unstale":"2026-02-13T02:22:18.051246+0000","last_undegraded":"2026-02-13T02:22:18.051246+0000","last_fullsized":"2026-02-13T02:22:18.051246+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T04:34:31.290868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047168+0000","last_change":"2026-02-13T02:22:17.013289+0000","last_active":"2026-02-13T02:22:18.047168+0000","last_peered":"2026-02-13T02:22:18.047168+0000","last_clean":"2026-02-13T02:22:18.047168+0000","last_became_active":"2026-02-13T02:22:17.013160+0000","last_became_peered":"2026-02-13T02:22:17.013160+0000","last_unstale":"2026-02-13T02:22:18.047168+0000","last_undegraded":"2026-02-13T02:22:18.047168+0000","last_fullsized":"2026-02-13T02:22:18.047168+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:19:06.942113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047197+0000","last_change":"2026-02-13T02:22:16.007420+0000","last_active":"2026-02-13T02:22:18.047197+0000","last_peered":"2026-02-13T02:22:18.047197+0000","last_clean":"2026-02-13T02:22:18.047197+0000","last_became_active":"2026-02-13T02:22:16.007114+0000","last_became_peered":"2026-02-13T02:22:16.007114+0000","last_unstale":"2026-02-13T02:22:18.047197+0000","last_undegraded":"2026-02-13T02:22:18.047197+0000","last_fullsized":"2026-02-13T02:22:18.047197+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:54:40.155679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051284+0000","last_change":"2026-02-13T02:22:17.021679+0000","last_active":"2026-02-13T02:22:18.051284+0000","last_peered":"2026-02-13T02:22:18.051284+0000","last_clean":"2026-02-13T02:22:18.051284+0000","last_became_active":"2026-02-13T02:22:17.021041+0000","last_became_peered":"2026-02-13T02:22:17.021041+0000","last_unstale":"2026-02-13T02:22:18.051284+0000","last_undegraded":"2026-02-13T02:22:18.051284+0000","last_fullsized":"2026-02-13T02:22:18.051284+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T04:37:08.611062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051417+0000","last_change":"2026-02-13T02:22:16.016749+0000","last_active":"2026-02-13T02:22:18.051417+0000","last_peered":"2026-02-13T02:22:18.051417+0000","last_clean":"2026-02-13T02:22:18.051417+0000","last_became_active":"2026-02-13T02:22:16.016620+0000","last_became_peered":"2026-02-13T02:22:16.016620+0000","last_unstale":"2026-02-13T02:22:18.051417+0000","last_undegraded":"2026-02-13T02:22:18.051417+0000","last_fullsized":"2026-02-13T02:22:18.051417+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:29:12.370221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051646+0000","last_change":"2026-02-13T02:22:17.025708+0000","last_active":"2026-02-13T02:22:18.051646+0000","last_peered":"2026-02-13T02:22:18.051646+0000","last_clean":"2026-02-13T02:22:18.051646+0000","last_became_active":"2026-02-13T02:22:17.025240+0000","last_became_peered":"2026-02-13T02:22:17.025240+0000","last_unstale":"2026-02-13T02:22:18.051646+0000","last_undegraded":"2026-02-13T02:22:18.051646+0000","last_fullsized":"2026-02-13T02:22:18.051646+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:04:02.085370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.011935+0000","last_change":"2026-02-13T02:22:16.009592+0000","last_active":"2026-02-13T02:22:18.011935+0000","last_peered":"2026-02-13T02:22:18.011935+0000","last_clean":"2026-02-13T02:22:18.011935+0000","last_became_active":"2026-02-13T02:22:16.009465+0000","last_became_peered":"2026-02-13T02:22:16.009465+0000","last_unstale":"2026-02-13T02:22:18.011935+0000","last_undegraded":"2026-02-13T02:22:18.011935+0000","last_fullsized":"2026-02-13T02:22:18.011935+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T02:33:29.133373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"70'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051885+0000","last_change":"2026-02-13T02:22:17.023729+0000","last_active":"2026-02-13T02:22:18.051885+0000","last_peered":"2026-02-13T02:22:18.051885+0000","last_clean":"2026-02-13T02:22:18.051885+0000","last_became_active":"2026-02-13T02:22:17.022007+0000","last_became_peered":"2026-02-13T02:22:17.022007+0000","last_unstale":"2026-02-13T02:22:18.051885+0000","last_undegraded":"2026-02-13T02:22:18.051885+0000","last_fullsized":"2026-02-13T02:22:18.051885+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T10:34:43.688805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.025077+0000","last_change":"2026-02-13T02:22:16.012023+0000","last_active":"2026-02-13T02:22:18.025077+0000","last_peered":"2026-02-13T02:22:18.025077+0000","last_clean":"2026-02-13T02:22:18.025077+0000","last_became_active":"2026-02-13T02:22:16.011680+0000","last_became_peered":"2026-02-13T02:22:16.011680+0000","last_unstale":"2026-02-13T02:22:18.025077+0000","last_undegraded":"2026-02-13T02:22:18.025077+0000","last_fullsized":"2026-02-13T02:22:18.025077+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T02:28:59.078471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047450+0000","last_change":"2026-02-13T02:22:17.015877+0000","last_active":"2026-02-13T02:22:18.047450+0000","last_peered":"2026-02-13T02:22:18.047450+0000","last_clean":"2026-02-13T02:22:18.047450+0000","last_became_active":"2026-02-13T02:22:17.015701+0000","last_became_peered":"2026-02-13T02:22:17.015701+0000","last_unstale":"2026-02-13T02:22:18.047450+0000","last_undegraded":"2026-02-13T02:22:18.047450+0000","last_fullsized":"2026-02-13T02:22:18.047450+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:35:45.903754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"70'4","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:23.020808+0000","last_change":"2026-02-13T02:22:16.008749+0000","last_active":"2026-02-13T02:22:23.020808+0000","last_peered":"2026-02-13T02:22:23.020808+0000","last_clean":"2026-02-13T02:22:23.020808+0000","last_became_active":"2026-02-13T02:22:16.008597+0000","last_became_peered":"2026-02-13T02:22:16.008597+0000","last_unstale":"2026-02-13T02:22:23.020808+0000","last_undegraded":"2026-02-13T02:22:23.020808+0000","last_fullsized":"2026-02-13T02:22:23.020808+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T10:18:25.963625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.021323+0000","last_change":"2026-02-13T02:22:17.017797+0000","last_active":"2026-02-13T02:22:18.021323+0000","last_peered":"2026-02-13T02:22:18.021323+0000","last_clean":"2026-02-13T02:22:18.021323+0000","last_became_active":"2026-02-13T02:22:17.017174+0000","last_became_peered":"2026-02-13T02:22:17.017174+0000","last_unstale":"2026-02-13T02:22:18.021323+0000","last_undegraded":"2026-02-13T02:22:18.021323+0000","last_fullsized":"2026-02-13T02:22:18.021323+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:39:41.102807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"69'2","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016292+0000","last_change":"2026-02-13T02:22:16.017646+0000","last_active":"2026-02-13T02:22:18.016292+0000","last_peered":"2026-02-13T02:22:18.016292+0000","last_clean":"2026-02-13T02:22:18.016292+0000","last_became_active":"2026-02-13T02:22:16.017529+0000","last_became_peered":"2026-02-13T02:22:16.017529+0000","last_unstale":"2026-02-13T02:22:18.016292+0000","last_undegraded":"2026-02-13T02:22:18.016292+0000","last_fullsized":"2026-02-13T02:22:18.016292+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T12:34:48.977200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047594+0000","last_change":"2026-02-13T02:22:17.019503+0000","last_active":"2026-02-13T02:22:18.047594+0000","last_peered":"2026-02-13T02:22:18.047594+0000","last_clean":"2026-02-13T02:22:18.047594+0000","last_became_active":"2026-02-13T02:22:17.019259+0000","last_became_peered":"2026-02-13T02:22:17.019259+0000","last_unstale":"2026-02-13T02:22:18.047594+0000","last_undegraded":"2026-02-13T02:22:18.047594+0000","last_fullsized":"2026-02-13T02:22:18.047594+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:19:49.339245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.050971+0000","last_change":"2026-02-13T02:22:16.016248+0000","last_active":"2026-02-13T02:22:18.050971+0000","last_peered":"2026-02-13T02:22:18.050971+0000","last_clean":"2026-02-13T02:22:18.050971+0000","last_became_active":"2026-02-13T02:22:16.016016+0000","last_became_peered":"2026-02-13T02:22:16.016016+0000","last_unstale":"2026-02-13T02:22:18.050971+0000","last_undegraded":"2026-02-13T02:22:18.050971+0000","last_fullsized":"2026-02-13T02:22:18.050971+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:39:11.300311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.024643+0000","last_change":"2026-02-13T02:22:17.023656+0000","last_active":"2026-02-13T02:22:18.024643+0000","last_peered":"2026-02-13T02:22:18.024643+0000","last_clean":"2026-02-13T02:22:18.024643+0000","last_became_active":"2026-02-13T02:22:17.023524+0000","last_became_peered":"2026-02-13T02:22:17.023524+0000","last_unstale":"2026-02-13T02:22:18.024643+0000","last_undegraded":"2026-02-13T02:22:18.024643+0000","last_fullsized":"2026-02-13T02:22:18.024643+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:33:34.390446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.020934+0000","last_change":"2026-02-13T02:22:16.010000+0000","last_active":"2026-02-13T02:22:18.020934+0000","last_peered":"2026-02-13T02:22:18.020934+0000","last_clean":"2026-02-13T02:22:18.020934+0000","last_became_active":"2026-02-13T02:22:16.009736+0000","last_became_peered":"2026-02-13T02:22:16.009736+0000","last_unstale":"2026-02-13T02:22:18.020934+0000","last_undegraded":"2026-02-13T02:22:18.020934+0000","last_fullsized":"2026-02-13T02:22:18.020934+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:16:46.363188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016302+0000","last_change":"2026-02-13T02:22:17.022375+0000","last_active":"2026-02-13T02:22:18.016302+0000","last_peered":"2026-02-13T02:22:18.016302+0000","last_clean":"2026-02-13T02:22:18.016302+0000","last_became_active":"2026-02-13T02:22:17.022236+0000","last_became_peered":"2026-02-13T02:22:17.022236+0000","last_unstale":"2026-02-13T02:22:18.016302+0000","last_undegraded":"2026-02-13T02:22:18.016302+0000","last_fullsized":"2026-02-13T02:22:18.016302+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:32:39.302116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.015763+0000","last_change":"2026-02-13T02:22:16.015222+0000","last_active":"2026-02-13T02:22:18.015763+0000","last_peered":"2026-02-13T02:22:18.015763+0000","last_clean":"2026-02-13T02:22:18.015763+0000","last_became_active":"2026-02-13T02:22:16.014947+0000","last_became_peered":"2026-02-13T02:22:16.014947+0000","last_unstale":"2026-02-13T02:22:18.015763+0000","last_undegraded":"2026-02-13T02:22:18.015763+0000","last_fullsized":"2026-02-13T02:22:18.015763+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T14:01:02.677585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.021118+0000","last_change":"2026-02-13T02:22:17.021948+0000","last_active":"2026-02-13T02:22:18.021118+0000","last_peered":"2026-02-13T02:22:18.021118+0000","last_clean":"2026-02-13T02:22:18.021118+0000","last_became_active":"2026-02-13T02:22:17.021764+0000","last_became_peered":"2026-02-13T02:22:17.021764+0000","last_unstale":"2026-02-13T02:22:18.021118+0000","last_undegraded":"2026-02-13T02:22:18.021118+0000","last_fullsized":"2026-02-13T02:22:18.021118+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:39:50.788814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"69'2","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.024813+0000","last_change":"2026-02-13T02:22:16.014740+0000","last_active":"2026-02-13T02:22:18.024813+0000","last_peered":"2026-02-13T02:22:18.024813+0000","last_clean":"2026-02-13T02:22:18.024813+0000","last_became_active":"2026-02-13T02:22:16.014578+0000","last_became_peered":"2026-02-13T02:22:16.014578+0000","last_unstale":"2026-02-13T02:22:18.024813+0000","last_undegraded":"2026-02-13T02:22:18.024813+0000","last_fullsized":"2026-02-13T02:22:18.024813+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:09:10.619351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051553+0000","last_change":"2026-02-13T02:22:17.025211+0000","last_active":"2026-02-13T02:22:18.051553+0000","last_peered":"2026-02-13T02:22:18.051553+0000","last_clean":"2026-02-13T02:22:18.051553+0000","last_became_active":"2026-02-13T02:22:17.025015+0000","last_became_peered":"2026-02-13T02:22:17.025015+0000","last_unstale":"2026-02-13T02:22:18.051553+0000","last_undegraded":"2026-02-13T02:22:18.051553+0000","last_fullsized":"2026-02-13T02:22:18.051553+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T04:28:08.456016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051394+0000","last_change":"2026-02-13T02:22:17.022434+0000","last_active":"2026-02-13T02:22:18.051394+0000","last_peered":"2026-02-13T02:22:18.051394+0000","last_clean":"2026-02-13T02:22:18.051394+0000","last_became_active":"2026-02-13T02:22:17.022160+0000","last_became_peered":"2026-02-13T02:22:17.022160+0000","last_unstale":"2026-02-13T02:22:18.051394+0000","last_undegraded":"2026-02-13T02:22:18.051394+0000","last_fullsized":"2026-02-13T02:22:18.051394+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T12:27:17.994699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"69'2","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.024901+0000","last_change":"2026-02-13T02:22:16.013867+0000","last_active":"2026-02-13T02:22:18.024901+0000","last_peered":"2026-02-13T02:22:18.024901+0000","last_clean":"2026-02-13T02:22:18.024901+0000","last_became_active":"2026-02-13T02:22:16.013689+0000","last_became_peered":"2026-02-13T02:22:16.013689+0000","last_unstale":"2026-02-13T02:22:18.024901+0000","last_undegraded":"2026-02-13T02:22:18.024901+0000","last_fullsized":"2026-02-13T02:22:18.024901+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T04:46:39.476015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051182+0000","last_change":"2026-02-13T02:22:17.022407+0000","last_active":"2026-02-13T02:22:18.051182+0000","last_peered":"2026-02-13T02:22:18.051182+0000","last_clean":"2026-02-13T02:22:18.051182+0000","last_became_active":"2026-02-13T02:22:17.021354+0000","last_became_peered":"2026-02-13T02:22:17.021354+0000","last_unstale":"2026-02-13T02:22:18.051182+0000","last_undegraded":"2026-02-13T02:22:18.051182+0000","last_fullsized":"2026-02-13T02:22:18.051182+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:35:18.694829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"69'2","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.025017+0000","last_change":"2026-02-13T02:22:16.015538+0000","last_active":"2026-02-13T02:22:18.025017+0000","last_peered":"2026-02-13T02:22:18.025017+0000","last_clean":"2026-02-13T02:22:18.025017+0000","last_became_active":"2026-02-13T02:22:16.015441+0000","last_became_peered":"2026-02-13T02:22:16.015441+0000","last_unstale":"2026-02-13T02:22:18.025017+0000","last_undegraded":"2026-02-13T02:22:18.025017+0000","last_fullsized":"2026-02-13T02:22:18.025017+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T02:34:39.849666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047574+0000","last_change":"2026-02-13T02:22:17.018483+0000","last_active":"2026-02-13T02:22:18.047574+0000","last_peered":"2026-02-13T02:22:18.047574+0000","last_clean":"2026-02-13T02:22:18.047574+0000","last_became_active":"2026-02-13T02:22:17.018318+0000","last_became_peered":"2026-02-13T02:22:17.018318+0000","last_unstale":"2026-02-13T02:22:18.047574+0000","last_undegraded":"2026-02-13T02:22:18.047574+0000","last_fullsized":"2026-02-13T02:22:18.047574+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T10:22:49.603621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"69'1","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.021277+0000","last_change":"2026-02-13T02:22:16.008795+0000","last_active":"2026-02-13T02:22:18.021277+0000","last_peered":"2026-02-13T02:22:18.021277+0000","last_clean":"2026-02-13T02:22:18.021277+0000","last_became_active":"2026-02-13T02:22:16.008621+0000","last_became_peered":"2026-02-13T02:22:16.008621+0000","last_unstale":"2026-02-13T02:22:18.021277+0000","last_undegraded":"2026-02-13T02:22:18.021277+0000","last_fullsized":"2026-02-13T02:22:18.021277+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:43:45.861077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.021240+0000","last_change":"2026-02-13T02:22:17.019826+0000","last_active":"2026-02-13T02:22:18.021240+0000","last_peered":"2026-02-13T02:22:18.021240+0000","last_clean":"2026-02-13T02:22:18.021240+0000","last_became_active":"2026-02-13T02:22:17.019429+0000","last_became_peered":"2026-02-13T02:22:17.019429+0000","last_unstale":"2026-02-13T02:22:18.021240+0000","last_undegraded":"2026-02-13T02:22:18.021240+0000","last_fullsized":"2026-02-13T02:22:18.021240+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:52:59.597088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"69'4","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:19.205618+0000","last_change":"2026-02-13T02:22:16.011002+0000","last_active":"2026-02-13T02:22:19.205618+0000","last_peered":"2026-02-13T02:22:19.205618+0000","last_clean":"2026-02-13T02:22:19.205618+0000","last_became_active":"2026-02-13T02:22:16.010892+0000","last_became_peered":"2026-02-13T02:22:16.010892+0000","last_unstale":"2026-02-13T02:22:19.205618+0000","last_undegraded":"2026-02-13T02:22:19.205618+0000","last_fullsized":"2026-02-13T02:22:19.205618+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:57:22.333777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":604,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":53664,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":27,"num_read_kb":68,"num_write":65,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":160992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":69115,"internal_metadata":0},"log_size":56,"ondisk_log_size":56,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":189,"num_write_kb":3716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":264161,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":65,"seq":279172874247,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27696,"kb_used_data":732,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":733975504,"statfs":{"total":751619276800,"available":751590916096,"internally_reserved":0,"allocated":749568,"data_stored":364081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8466,"internal_metadata":27582190},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":58,"seq":249108103177,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27820,"kb_used_data":856,"kb_used_omap":18,"kb_used_meta":26925,"kb_avail":733975380,"statfs":{"total":751619276800,"available":751590789120,"internally_reserved":0,"allocated":876544,"data_stored":1007604,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":19356,"internal_metadata":27571300},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":54,"seq":231928233995,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27700,"kb_used_data":736,"kb_used_omap":13,"kb_used_meta":26930,"kb_avail":733975500,"statfs":{"total":751619276800,"available":751590912000,"internally_reserved":0,"allocated":753664,"data_stored":364685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13407,"internal_metadata":27577249},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":47,"seq":201863462924,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27704,"kb_used_data":740,"kb_used_omap":21,"kb_used_meta":26922,"kb_avail":733975496,"statfs":{"total":751619276800,"available":751590907904,"internally_reserved":0,"allocated":757760,"data_stored":364707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22372,"internal_metadata":27568284},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":40,"seq":171798691853,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27752,"kb_used_data":788,"kb_used_omap":23,"kb_used_meta":26920,"kb_avail":733975448,"statfs":{"total":751619276800,"available":751590858752,"internally_reserved":0,"allocated":806912,"data_stored":955170,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":24500,"internal_metadata":27566156},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":37,"seq":158913789968,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27700,"kb_used_data":736,"kb_used_omap":23,"kb_used_meta":26920,"kb_avail":733975500,"statfs":{"total":751619276800,"available":751590912000,"internally_reserved":0,"allocated":753664,"data_stored":364198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24460,"internal_metadata":27566196},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":30,"seq":128849018897,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27760,"kb_used_data":796,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":733975440,"statfs":{"total":751619276800,"available":751590850560,"internally_reserved":0,"allocated":815104,"data_stored":954748,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":29636,"internal_metadata":27561020},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":26,"seq":111669149715,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27700,"kb_used_data":736,"kb_used_omap":23,"kb_used_meta":26920,"kb_avail":733975500,"statfs":{"total":751619276800,"available":751590912000,"internally_reserved":0,"allocated":753664,"data_stored":364103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24268,"internal_metadata":27566388},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":19,"seq":81604378644,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27712,"kb_used_data":748,"kb_used_omap":40,"kb_used_meta":26903,"kb_avail":733975488,"statfs":{"total":751619276800,"available":751590899712,"internally_reserved":0,"allocated":765952,"data_stored":364419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41896,"internal_metadata":27548760},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27760,"kb_used_data":796,"kb_used_omap":48,"kb_used_meta":26895,"kb_avail":733975440,"statfs":{"total":751619276800,"available":751590850560,"internally_reserved":0,"allocated":815104,"data_stored":416959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49734,"internal_metadata":27540922},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672983,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27752,"kb_used_data":788,"kb_used_omap":38,"kb_used_meta":26905,"kb_avail":733975448,"statfs":{"total":751619276800,"available":751590858752,"internally_reserved":0,"allocated":806912,"data_stored":411609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39406,"internal_metadata":27551250},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":7,"seq":30064771096,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27700,"kb_used_data":736,"kb_used_omap":40,"kb_used_meta":26903,"kb_avail":733975500,"statfs":{"total":751619276800,"available":751590912000,"internally_reserved":0,"allocated":753664,"data_stored":364171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41083,"internal_metadata":27549573},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":36563,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":36563,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20097,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":52844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":705,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":52878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13826,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1837,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1258,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":366,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1586,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":656,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5747,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1290,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":53038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17757,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3990,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-13T02:22:30.947 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph pg dump --format=json 2026-02-13T02:22:31.095 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:31.207 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.204+0000 7f7338dc46c0 1 -- 10.20.193.27:0/654406695 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7334115440 msgr2=0x7f7334175de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:31.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.204+0000 7f7338dc46c0 1 --2- 10.20.193.27:0/654406695 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7334115440 0x7f7334175de0 secure :-1 s=READY pgs=332 gs=1 cs=0 l=1 c_cookie=8740849805703203757 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f732400aa70 tx=0x7f732402fde0 comp rx=0 tx=0).stop 2026-02-13T02:22:31.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.204+0000 7f7338dc46c0 1 -- 10.20.193.27:0/654406695 shutdown_connections 2026-02-13T02:22:31.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.204+0000 7f7338dc46c0 1 --2- 10.20.193.27:0/654406695 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7334115440 0x7f7334175de0 unknown :-1 s=CLOSED pgs=332 gs=1 cs=0 l=1 c_cookie=8740849805703203757 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:31.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.204+0000 7f7338dc46c0 1 --2- 10.20.193.27:0/654406695 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f733410fc20 0x7f7334114c60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:31.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.204+0000 7f7338dc46c0 1 --2- 10.20.193.27:0/654406695 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f733410f230 0x7f733410f600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:31.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.204+0000 7f7338dc46c0 1 -- 10.20.193.27:0/654406695 >> 10.20.193.27:0/654406695 conn(0x7f73340ecb30 msgr2=0x7f733410cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:31.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.204+0000 7f7338dc46c0 1 -- 10.20.193.27:0/654406695 shutdown_connections 2026-02-13T02:22:31.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.204+0000 7f7338dc46c0 1 -- 10.20.193.27:0/654406695 wait complete. 2026-02-13T02:22:31.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.205+0000 7f7338dc46c0 1 Processor -- start 2026-02-13T02:22:31.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.205+0000 7f7338dc46c0 1 -- start start 2026-02-13T02:22:31.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.205+0000 7f7338dc46c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f733410f230 0x7f733411d1e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:31.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.205+0000 7f7338dc46c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f7334115440 0x7f733411d720 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:31.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.205+0000 7f7338dc46c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7334122ae0 0x7f7334122f80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:31.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.205+0000 7f7338dc46c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f73341234c0 con 0x7f733410f230 2026-02-13T02:22:31.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.205+0000 7f7338dc46c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7334123630 con 0x7f7334122ae0 2026-02-13T02:22:31.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.205+0000 7f7338dc46c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f73341237f0 con 0x7f7334115440 2026-02-13T02:22:31.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.205+0000 7f73325756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f733410f230 0x7f733411d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:31.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.205+0000 7f73325756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f733410f230 0x7f733411d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:54020/0 (socket says 10.20.193.27:54020) 2026-02-13T02:22:31.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.205+0000 7f73325756c0 1 -- 10.20.193.27:0/2241086738 learned_addr learned my addr 10.20.193.27:0/2241086738 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:31.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.205+0000 7f7331d746c0 1 --2- 10.20.193.27:0/2241086738 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f7334115440 0x7f733411d720 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:31.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.205+0000 7f7332d766c0 1 --2- 10.20.193.27:0/2241086738 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7334122ae0 0x7f7334122f80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:31.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.205+0000 7f73325756c0 1 -- 10.20.193.27:0/2241086738 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f7334115440 msgr2=0x7f733411d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:31.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.205+0000 7f73325756c0 1 --2- 10.20.193.27:0/2241086738 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f7334115440 0x7f733411d720 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:31.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.205+0000 7f73325756c0 1 -- 10.20.193.27:0/2241086738 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7334122ae0 msgr2=0x7f7334122f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:31.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.205+0000 7f73325756c0 1 --2- 10.20.193.27:0/2241086738 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7334122ae0 0x7f7334122f80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:31.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.205+0000 7f73325756c0 1 -- 10.20.193.27:0/2241086738 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732400a660 con 0x7f733410f230 2026-02-13T02:22:31.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.205+0000 7f7331d746c0 1 --2- 10.20.193.27:0/2241086738 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f7334115440 0x7f733411d720 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:22:31.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.205+0000 7f73325756c0 1 --2- 10.20.193.27:0/2241086738 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f733410f230 0x7f733411d1e0 secure :-1 s=READY pgs=333 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7328009490 tx=0x7f732800bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:31.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.205+0000 7f731b7fe6c0 1 -- 10.20.193.27:0/2241086738 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f7328004a80 con 0x7f733410f230 2026-02-13T02:22:31.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.205+0000 7f7338dc46c0 1 -- 10.20.193.27:0/2241086738 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7334163c70 con 0x7f733410f230 2026-02-13T02:22:31.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.205+0000 7f731b7fe6c0 1 -- 10.20.193.27:0/2241086738 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f7328009030 con 0x7f733410f230 2026-02-13T02:22:31.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.205+0000 7f731b7fe6c0 1 -- 10.20.193.27:0/2241086738 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f732800ce30 con 0x7f733410f230 2026-02-13T02:22:31.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.205+0000 7f7338dc46c0 1 -- 10.20.193.27:0/2241086738 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73341641a0 con 0x7f733410f230 2026-02-13T02:22:31.211 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.206+0000 7f7338dc46c0 1 -- 10.20.193.27:0/2241086738 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7334110500 con 0x7f733410f230 2026-02-13T02:22:31.212 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.207+0000 7f731b7fe6c0 1 -- 10.20.193.27:0/2241086738 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f7328004060 con 0x7f733410f230 2026-02-13T02:22:31.212 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.208+0000 7f731b7fe6c0 1 --2- 10.20.193.27:0/2241086738 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f730c0b5690 0x7f730c0d5b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:31.212 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.208+0000 7f731b7fe6c0 1 -- 10.20.193.27:0/2241086738 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f73280e3b60 con 0x7f733410f230 2026-02-13T02:22:31.212 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.208+0000 7f7331d746c0 1 --2- 10.20.193.27:0/2241086738 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f730c0b5690 0x7f730c0d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:31.212 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.208+0000 7f731b7fe6c0 1 -- 10.20.193.27:0/2241086738 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f73280a6580 con 0x7f733410f230 2026-02-13T02:22:31.212 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.208+0000 7f7331d746c0 1 --2- 10.20.193.27:0/2241086738 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f730c0b5690 0x7f730c0d5b40 secure :-1 s=READY pgs=230 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7334123b70 tx=0x7f731c005e50 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:31.299 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.298+0000 7f7338dc46c0 1 -- 10.20.193.27:0/2241086738 --> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f733411e330 con 0x7f730c0b5690 2026-02-13T02:22:31.301 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.300+0000 7f731b7fe6c0 1 -- 10.20.193.27:0/2241086738 <== mgr.14116 v2:10.20.193.27:6800/1837452956 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258878 (secure 0 0 0) 0x7f733411e330 con 0x7f730c0b5690 2026-02-13T02:22:31.304 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:22:31.306 INFO:teuthology.orchestra.run.trial027.stderr:dumped all 2026-02-13T02:22:31.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.301+0000 7f7338dc46c0 1 -- 10.20.193.27:0/2241086738 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f730c0b5690 msgr2=0x7f730c0d5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:31.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.301+0000 7f7338dc46c0 1 --2- 10.20.193.27:0/2241086738 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f730c0b5690 0x7f730c0d5b40 secure :-1 s=READY pgs=230 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7334123b70 tx=0x7f731c005e50 comp rx=0 tx=0).stop 2026-02-13T02:22:31.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.301+0000 7f7338dc46c0 1 -- 10.20.193.27:0/2241086738 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f733410f230 msgr2=0x7f733411d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:31.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.301+0000 7f7338dc46c0 1 --2- 10.20.193.27:0/2241086738 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f733410f230 0x7f733411d1e0 secure :-1 s=READY pgs=333 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7328009490 tx=0x7f732800bca0 comp rx=0 tx=0).stop 2026-02-13T02:22:31.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.301+0000 7f7338dc46c0 1 -- 10.20.193.27:0/2241086738 shutdown_connections 2026-02-13T02:22:31.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.301+0000 7f7338dc46c0 1 --2- 10.20.193.27:0/2241086738 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f730c0b5690 0x7f730c0d5b40 unknown :-1 s=CLOSED pgs=230 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:31.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.301+0000 7f7338dc46c0 1 --2- 10.20.193.27:0/2241086738 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7334122ae0 0x7f7334122f80 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:31.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.301+0000 7f7338dc46c0 1 --2- 10.20.193.27:0/2241086738 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f7334115440 0x7f733411d720 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:31.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.301+0000 7f7338dc46c0 1 --2- 10.20.193.27:0/2241086738 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f733410f230 0x7f733411d1e0 unknown :-1 s=CLOSED pgs=333 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:31.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.301+0000 7f7338dc46c0 1 -- 10.20.193.27:0/2241086738 >> 10.20.193.27:0/2241086738 conn(0x7f73340ecb30 msgr2=0x7f733410cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:31.309 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.301+0000 7f7338dc46c0 1 -- 10.20.193.27:0/2241086738 shutdown_connections 2026-02-13T02:22:31.309 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.301+0000 7f7338dc46c0 1 -- 10.20.193.27:0/2241086738 wait complete. 2026-02-13T02:22:31.313 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:31 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/213258991' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2026-02-13T02:22:31.313 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:31 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/953242410' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2026-02-13T02:22:31.313 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:31 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/4036026419' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2026-02-13T02:22:31.313 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:31 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2385519176' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2026-02-13T02:22:31.313 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:31 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2319221594' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2026-02-13T02:22:31.313 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:31 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/158182147' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2026-02-13T02:22:31.313 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:31 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3563207069' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2026-02-13T02:22:31.313 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:31 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2697967832' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2026-02-13T02:22:31.343 INFO:teuthology.orchestra.run.trial027.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2026-02-13T02:22:30.958937+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644266,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":133,"num_read_kb":159,"num_write":260,"num_write_kb":3797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":8808038400,"kb_used":332756,"kb_used_data":9188,"kb_used_omap":331,"kb_used_meta":322996,"kb_avail":8807705644,"statfs":{"total":9019431321600,"available":9019090579456,"internally_reserved":0,"allocated":9408512,"data_stored":6296454,"data_compressed":114621,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":339071,"internal_metadata":330748801},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":53864,"num_objects":20,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":27,"num_read_kb":68,"num_write":62,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002700"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016636+0000","last_change":"2026-02-13T02:22:17.023288+0000","last_active":"2026-02-13T02:22:18.016636+0000","last_peered":"2026-02-13T02:22:18.016636+0000","last_clean":"2026-02-13T02:22:18.016636+0000","last_became_active":"2026-02-13T02:22:17.023073+0000","last_became_peered":"2026-02-13T02:22:17.023073+0000","last_unstale":"2026-02-13T02:22:18.016636+0000","last_undegraded":"2026-02-13T02:22:18.016636+0000","last_fullsized":"2026-02-13T02:22:18.016636+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:19:00.322079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047477+0000","last_change":"2026-02-13T02:22:17.018633+0000","last_active":"2026-02-13T02:22:18.047477+0000","last_peered":"2026-02-13T02:22:18.047477+0000","last_clean":"2026-02-13T02:22:18.047477+0000","last_became_active":"2026-02-13T02:22:17.018516+0000","last_became_peered":"2026-02-13T02:22:17.018516+0000","last_unstale":"2026-02-13T02:22:18.047477+0000","last_undegraded":"2026-02-13T02:22:18.047477+0000","last_fullsized":"2026-02-13T02:22:18.047477+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:33:08.892207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047500+0000","last_change":"2026-02-13T02:22:17.019231+0000","last_active":"2026-02-13T02:22:18.047500+0000","last_peered":"2026-02-13T02:22:18.047500+0000","last_clean":"2026-02-13T02:22:18.047500+0000","last_became_active":"2026-02-13T02:22:17.019039+0000","last_became_peered":"2026-02-13T02:22:17.019039+0000","last_unstale":"2026-02-13T02:22:18.047500+0000","last_undegraded":"2026-02-13T02:22:18.047500+0000","last_fullsized":"2026-02-13T02:22:18.047500+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:21:31.573485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016292+0000","last_change":"2026-02-13T02:22:17.021960+0000","last_active":"2026-02-13T02:22:18.016292+0000","last_peered":"2026-02-13T02:22:18.016292+0000","last_clean":"2026-02-13T02:22:18.016292+0000","last_became_active":"2026-02-13T02:22:17.021325+0000","last_became_peered":"2026-02-13T02:22:17.021325+0000","last_unstale":"2026-02-13T02:22:18.016292+0000","last_undegraded":"2026-02-13T02:22:18.016292+0000","last_fullsized":"2026-02-13T02:22:18.016292+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:28:01.069244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.024818+0000","last_change":"2026-02-13T02:22:17.022417+0000","last_active":"2026-02-13T02:22:18.024818+0000","last_peered":"2026-02-13T02:22:18.024818+0000","last_clean":"2026-02-13T02:22:18.024818+0000","last_became_active":"2026-02-13T02:22:17.022150+0000","last_became_peered":"2026-02-13T02:22:17.022150+0000","last_unstale":"2026-02-13T02:22:18.024818+0000","last_undegraded":"2026-02-13T02:22:18.024818+0000","last_fullsized":"2026-02-13T02:22:18.024818+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T12:26:21.518119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051355+0000","last_change":"2026-02-13T02:22:17.024197+0000","last_active":"2026-02-13T02:22:18.051355+0000","last_peered":"2026-02-13T02:22:18.051355+0000","last_clean":"2026-02-13T02:22:18.051355+0000","last_became_active":"2026-02-13T02:22:17.023378+0000","last_became_peered":"2026-02-13T02:22:17.023378+0000","last_unstale":"2026-02-13T02:22:18.051355+0000","last_undegraded":"2026-02-13T02:22:18.051355+0000","last_fullsized":"2026-02-13T02:22:18.051355+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:17:40.942731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051199+0000","last_change":"2026-02-13T02:22:17.021184+0000","last_active":"2026-02-13T02:22:18.051199+0000","last_peered":"2026-02-13T02:22:18.051199+0000","last_clean":"2026-02-13T02:22:18.051199+0000","last_became_active":"2026-02-13T02:22:17.020961+0000","last_became_peered":"2026-02-13T02:22:17.020961+0000","last_unstale":"2026-02-13T02:22:18.051199+0000","last_undegraded":"2026-02-13T02:22:18.051199+0000","last_fullsized":"2026-02-13T02:22:18.051199+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T10:38:19.326702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.020937+0000","last_change":"2026-02-13T02:22:17.021948+0000","last_active":"2026-02-13T02:22:18.020937+0000","last_peered":"2026-02-13T02:22:18.020937+0000","last_clean":"2026-02-13T02:22:18.020937+0000","last_became_active":"2026-02-13T02:22:17.021760+0000","last_became_peered":"2026-02-13T02:22:17.021760+0000","last_unstale":"2026-02-13T02:22:18.020937+0000","last_undegraded":"2026-02-13T02:22:18.020937+0000","last_fullsized":"2026-02-13T02:22:18.020937+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:24:58.456659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051727+0000","last_change":"2026-02-13T02:22:17.024036+0000","last_active":"2026-02-13T02:22:18.051727+0000","last_peered":"2026-02-13T02:22:18.051727+0000","last_clean":"2026-02-13T02:22:18.051727+0000","last_became_active":"2026-02-13T02:22:17.023805+0000","last_became_peered":"2026-02-13T02:22:17.023805+0000","last_unstale":"2026-02-13T02:22:18.051727+0000","last_undegraded":"2026-02-13T02:22:18.051727+0000","last_fullsized":"2026-02-13T02:22:18.051727+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T02:44:29.738084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051866+0000","last_change":"2026-02-13T02:22:17.024557+0000","last_active":"2026-02-13T02:22:18.051866+0000","last_peered":"2026-02-13T02:22:18.051866+0000","last_clean":"2026-02-13T02:22:18.051866+0000","last_became_active":"2026-02-13T02:22:17.024270+0000","last_became_peered":"2026-02-13T02:22:17.024270+0000","last_unstale":"2026-02-13T02:22:18.051866+0000","last_undegraded":"2026-02-13T02:22:18.051866+0000","last_fullsized":"2026-02-13T02:22:18.051866+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:36:05.815116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051729+0000","last_change":"2026-02-13T02:22:17.023942+0000","last_active":"2026-02-13T02:22:18.051729+0000","last_peered":"2026-02-13T02:22:18.051729+0000","last_clean":"2026-02-13T02:22:18.051729+0000","last_became_active":"2026-02-13T02:22:17.023475+0000","last_became_peered":"2026-02-13T02:22:17.023475+0000","last_unstale":"2026-02-13T02:22:18.051729+0000","last_undegraded":"2026-02-13T02:22:18.051729+0000","last_fullsized":"2026-02-13T02:22:18.051729+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T12:17:44.718186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016376+0000","last_change":"2026-02-13T02:22:17.022471+0000","last_active":"2026-02-13T02:22:18.016376+0000","last_peered":"2026-02-13T02:22:18.016376+0000","last_clean":"2026-02-13T02:22:18.016376+0000","last_became_active":"2026-02-13T02:22:17.021894+0000","last_became_peered":"2026-02-13T02:22:17.021894+0000","last_unstale":"2026-02-13T02:22:18.016376+0000","last_undegraded":"2026-02-13T02:22:18.016376+0000","last_fullsized":"2026-02-13T02:22:18.016376+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:27:33.483596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047429+0000","last_change":"2026-02-13T02:22:17.017830+0000","last_active":"2026-02-13T02:22:18.047429+0000","last_peered":"2026-02-13T02:22:18.047429+0000","last_clean":"2026-02-13T02:22:18.047429+0000","last_became_active":"2026-02-13T02:22:17.017232+0000","last_became_peered":"2026-02-13T02:22:17.017232+0000","last_unstale":"2026-02-13T02:22:18.047429+0000","last_undegraded":"2026-02-13T02:22:18.047429+0000","last_fullsized":"2026-02-13T02:22:18.047429+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:40:47.725661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047355+0000","last_change":"2026-02-13T02:22:17.021764+0000","last_active":"2026-02-13T02:22:18.047355+0000","last_peered":"2026-02-13T02:22:18.047355+0000","last_clean":"2026-02-13T02:22:18.047355+0000","last_became_active":"2026-02-13T02:22:17.021609+0000","last_became_peered":"2026-02-13T02:22:17.021609+0000","last_unstale":"2026-02-13T02:22:18.047355+0000","last_undegraded":"2026-02-13T02:22:18.047355+0000","last_fullsized":"2026-02-13T02:22:18.047355+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T12:04:31.168034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047332+0000","last_change":"2026-02-13T02:22:17.019584+0000","last_active":"2026-02-13T02:22:18.047332+0000","last_peered":"2026-02-13T02:22:18.047332+0000","last_clean":"2026-02-13T02:22:18.047332+0000","last_became_active":"2026-02-13T02:22:17.019388+0000","last_became_peered":"2026-02-13T02:22:17.019388+0000","last_unstale":"2026-02-13T02:22:18.047332+0000","last_undegraded":"2026-02-13T02:22:18.047332+0000","last_fullsized":"2026-02-13T02:22:18.047332+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:33:16.084048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.050998+0000","last_change":"2026-02-13T02:22:17.023950+0000","last_active":"2026-02-13T02:22:18.050998+0000","last_peered":"2026-02-13T02:22:18.050998+0000","last_clean":"2026-02-13T02:22:18.050998+0000","last_became_active":"2026-02-13T02:22:17.023029+0000","last_became_peered":"2026-02-13T02:22:17.023029+0000","last_unstale":"2026-02-13T02:22:18.050998+0000","last_undegraded":"2026-02-13T02:22:18.050998+0000","last_fullsized":"2026-02-13T02:22:18.050998+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:14:25.119381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016670+0000","last_change":"2026-02-13T02:22:17.024675+0000","last_active":"2026-02-13T02:22:18.016670+0000","last_peered":"2026-02-13T02:22:18.016670+0000","last_clean":"2026-02-13T02:22:18.016670+0000","last_became_active":"2026-02-13T02:22:17.024460+0000","last_became_peered":"2026-02-13T02:22:17.024460+0000","last_unstale":"2026-02-13T02:22:18.016670+0000","last_undegraded":"2026-02-13T02:22:18.016670+0000","last_fullsized":"2026-02-13T02:22:18.016670+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T04:41:20.995149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.012018+0000","last_change":"2026-02-13T02:22:17.019261+0000","last_active":"2026-02-13T02:22:18.012018+0000","last_peered":"2026-02-13T02:22:18.012018+0000","last_clean":"2026-02-13T02:22:18.012018+0000","last_became_active":"2026-02-13T02:22:17.019084+0000","last_became_peered":"2026-02-13T02:22:17.019084+0000","last_unstale":"2026-02-13T02:22:18.012018+0000","last_undegraded":"2026-02-13T02:22:18.012018+0000","last_fullsized":"2026-02-13T02:22:18.012018+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:26:06.852915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016186+0000","last_change":"2026-02-13T02:22:17.021336+0000","last_active":"2026-02-13T02:22:18.016186+0000","last_peered":"2026-02-13T02:22:18.016186+0000","last_clean":"2026-02-13T02:22:18.016186+0000","last_became_active":"2026-02-13T02:22:17.020885+0000","last_became_peered":"2026-02-13T02:22:17.020885+0000","last_unstale":"2026-02-13T02:22:18.016186+0000","last_undegraded":"2026-02-13T02:22:18.016186+0000","last_fullsized":"2026-02-13T02:22:18.016186+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T14:05:29.982317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016382+0000","last_change":"2026-02-13T02:22:17.022900+0000","last_active":"2026-02-13T02:22:18.016382+0000","last_peered":"2026-02-13T02:22:18.016382+0000","last_clean":"2026-02-13T02:22:18.016382+0000","last_became_active":"2026-02-13T02:22:17.022753+0000","last_became_peered":"2026-02-13T02:22:17.022753+0000","last_unstale":"2026-02-13T02:22:18.016382+0000","last_undegraded":"2026-02-13T02:22:18.016382+0000","last_fullsized":"2026-02-13T02:22:18.016382+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:35:07.485606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047442+0000","last_change":"2026-02-13T02:22:17.018325+0000","last_active":"2026-02-13T02:22:18.047442+0000","last_peered":"2026-02-13T02:22:18.047442+0000","last_clean":"2026-02-13T02:22:18.047442+0000","last_became_active":"2026-02-13T02:22:17.018167+0000","last_became_peered":"2026-02-13T02:22:17.018167+0000","last_unstale":"2026-02-13T02:22:18.047442+0000","last_undegraded":"2026-02-13T02:22:18.047442+0000","last_fullsized":"2026-02-13T02:22:18.047442+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T14:09:03.876188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051256+0000","last_change":"2026-02-13T02:22:17.022546+0000","last_active":"2026-02-13T02:22:18.051256+0000","last_peered":"2026-02-13T02:22:18.051256+0000","last_clean":"2026-02-13T02:22:18.051256+0000","last_became_active":"2026-02-13T02:22:17.022064+0000","last_became_peered":"2026-02-13T02:22:17.022064+0000","last_unstale":"2026-02-13T02:22:18.051256+0000","last_undegraded":"2026-02-13T02:22:18.051256+0000","last_fullsized":"2026-02-13T02:22:18.051256+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:55:49.327407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.020899+0000","last_change":"2026-02-13T02:22:17.018592+0000","last_active":"2026-02-13T02:22:18.020899+0000","last_peered":"2026-02-13T02:22:18.020899+0000","last_clean":"2026-02-13T02:22:18.020899+0000","last_became_active":"2026-02-13T02:22:17.018368+0000","last_became_peered":"2026-02-13T02:22:17.018368+0000","last_unstale":"2026-02-13T02:22:18.020899+0000","last_undegraded":"2026-02-13T02:22:18.020899+0000","last_fullsized":"2026-02-13T02:22:18.020899+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:15:11.963709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051251+0000","last_change":"2026-02-13T02:22:17.023462+0000","last_active":"2026-02-13T02:22:18.051251+0000","last_peered":"2026-02-13T02:22:18.051251+0000","last_clean":"2026-02-13T02:22:18.051251+0000","last_became_active":"2026-02-13T02:22:17.023327+0000","last_became_peered":"2026-02-13T02:22:17.023327+0000","last_unstale":"2026-02-13T02:22:18.051251+0000","last_undegraded":"2026-02-13T02:22:18.051251+0000","last_fullsized":"2026-02-13T02:22:18.051251+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:42:27.909493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.021321+0000","last_change":"2026-02-13T02:22:17.021211+0000","last_active":"2026-02-13T02:22:18.021321+0000","last_peered":"2026-02-13T02:22:18.021321+0000","last_clean":"2026-02-13T02:22:18.021321+0000","last_became_active":"2026-02-13T02:22:17.021002+0000","last_became_peered":"2026-02-13T02:22:17.021002+0000","last_unstale":"2026-02-13T02:22:18.021321+0000","last_undegraded":"2026-02-13T02:22:18.021321+0000","last_fullsized":"2026-02-13T02:22:18.021321+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:23:00.242280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016321+0000","last_change":"2026-02-13T02:22:17.023595+0000","last_active":"2026-02-13T02:22:18.016321+0000","last_peered":"2026-02-13T02:22:18.016321+0000","last_clean":"2026-02-13T02:22:18.016321+0000","last_became_active":"2026-02-13T02:22:17.023480+0000","last_became_peered":"2026-02-13T02:22:17.023480+0000","last_unstale":"2026-02-13T02:22:18.016321+0000","last_undegraded":"2026-02-13T02:22:18.016321+0000","last_fullsized":"2026-02-13T02:22:18.016321+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:13:14.505114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016555+0000","last_change":"2026-02-13T02:22:17.021599+0000","last_active":"2026-02-13T02:22:18.016555+0000","last_peered":"2026-02-13T02:22:18.016555+0000","last_clean":"2026-02-13T02:22:18.016555+0000","last_became_active":"2026-02-13T02:22:17.021373+0000","last_became_peered":"2026-02-13T02:22:17.021373+0000","last_unstale":"2026-02-13T02:22:18.016555+0000","last_undegraded":"2026-02-13T02:22:18.016555+0000","last_fullsized":"2026-02-13T02:22:18.016555+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:35:13.390476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047383+0000","last_change":"2026-02-13T02:22:17.017507+0000","last_active":"2026-02-13T02:22:18.047383+0000","last_peered":"2026-02-13T02:22:18.047383+0000","last_clean":"2026-02-13T02:22:18.047383+0000","last_became_active":"2026-02-13T02:22:17.017211+0000","last_became_peered":"2026-02-13T02:22:17.017211+0000","last_unstale":"2026-02-13T02:22:18.047383+0000","last_undegraded":"2026-02-13T02:22:18.047383+0000","last_fullsized":"2026-02-13T02:22:18.047383+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:19:19.531222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047561+0000","last_change":"2026-02-13T02:22:17.019486+0000","last_active":"2026-02-13T02:22:18.047561+0000","last_peered":"2026-02-13T02:22:18.047561+0000","last_clean":"2026-02-13T02:22:18.047561+0000","last_became_active":"2026-02-13T02:22:17.019288+0000","last_became_peered":"2026-02-13T02:22:17.019288+0000","last_unstale":"2026-02-13T02:22:18.047561+0000","last_undegraded":"2026-02-13T02:22:18.047561+0000","last_fullsized":"2026-02-13T02:22:18.047561+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:51:14.055654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.015982+0000","last_change":"2026-02-13T02:22:17.022029+0000","last_active":"2026-02-13T02:22:18.015982+0000","last_peered":"2026-02-13T02:22:18.015982+0000","last_clean":"2026-02-13T02:22:18.015982+0000","last_became_active":"2026-02-13T02:22:17.021771+0000","last_became_peered":"2026-02-13T02:22:17.021771+0000","last_unstale":"2026-02-13T02:22:18.015982+0000","last_undegraded":"2026-02-13T02:22:18.015982+0000","last_fullsized":"2026-02-13T02:22:18.015982+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:15:51.936117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016131+0000","last_change":"2026-02-13T02:22:17.021554+0000","last_active":"2026-02-13T02:22:18.016131+0000","last_peered":"2026-02-13T02:22:18.016131+0000","last_clean":"2026-02-13T02:22:18.016131+0000","last_became_active":"2026-02-13T02:22:17.021248+0000","last_became_peered":"2026-02-13T02:22:17.021248+0000","last_unstale":"2026-02-13T02:22:18.016131+0000","last_undegraded":"2026-02-13T02:22:18.016131+0000","last_fullsized":"2026-02-13T02:22:18.016131+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T12:27:26.132715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016224+0000","last_change":"2026-02-13T02:22:17.022426+0000","last_active":"2026-02-13T02:22:18.016224+0000","last_peered":"2026-02-13T02:22:18.016224+0000","last_clean":"2026-02-13T02:22:18.016224+0000","last_became_active":"2026-02-13T02:22:17.022285+0000","last_became_peered":"2026-02-13T02:22:17.022285+0000","last_unstale":"2026-02-13T02:22:18.016224+0000","last_undegraded":"2026-02-13T02:22:18.016224+0000","last_fullsized":"2026-02-13T02:22:18.016224+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:25:58.151252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.024814+0000","last_change":"2026-02-13T02:22:17.022749+0000","last_active":"2026-02-13T02:22:18.024814+0000","last_peered":"2026-02-13T02:22:18.024814+0000","last_clean":"2026-02-13T02:22:18.024814+0000","last_became_active":"2026-02-13T02:22:17.022397+0000","last_became_peered":"2026-02-13T02:22:17.022397+0000","last_unstale":"2026-02-13T02:22:18.024814+0000","last_undegraded":"2026-02-13T02:22:18.024814+0000","last_fullsized":"2026-02-13T02:22:18.024814+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:28:44.768608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"69'1","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051275+0000","last_change":"2026-02-13T02:22:16.014721+0000","last_active":"2026-02-13T02:22:18.051275+0000","last_peered":"2026-02-13T02:22:18.051275+0000","last_clean":"2026-02-13T02:22:18.051275+0000","last_became_active":"2026-02-13T02:22:16.014587+0000","last_became_peered":"2026-02-13T02:22:16.014587+0000","last_unstale":"2026-02-13T02:22:18.051275+0000","last_undegraded":"2026-02-13T02:22:18.051275+0000","last_fullsized":"2026-02-13T02:22:18.051275+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:56:58.003775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051261+0000","last_change":"2026-02-13T02:22:17.022514+0000","last_active":"2026-02-13T02:22:18.051261+0000","last_peered":"2026-02-13T02:22:18.051261+0000","last_clean":"2026-02-13T02:22:18.051261+0000","last_became_active":"2026-02-13T02:22:17.022273+0000","last_became_peered":"2026-02-13T02:22:17.022273+0000","last_unstale":"2026-02-13T02:22:18.051261+0000","last_undegraded":"2026-02-13T02:22:18.051261+0000","last_fullsized":"2026-02-13T02:22:18.051261+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:33:27.805740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"69'1","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051311+0000","last_change":"2026-02-13T02:22:16.012690+0000","last_active":"2026-02-13T02:22:18.051311+0000","last_peered":"2026-02-13T02:22:18.051311+0000","last_clean":"2026-02-13T02:22:18.051311+0000","last_became_active":"2026-02-13T02:22:16.011935+0000","last_became_peered":"2026-02-13T02:22:16.011935+0000","last_unstale":"2026-02-13T02:22:18.051311+0000","last_undegraded":"2026-02-13T02:22:18.051311+0000","last_fullsized":"2026-02-13T02:22:18.051311+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T06:55:46.440839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047121+0000","last_change":"2026-02-13T02:22:16.007738+0000","last_active":"2026-02-13T02:22:18.047121+0000","last_peered":"2026-02-13T02:22:18.047121+0000","last_clean":"2026-02-13T02:22:18.047121+0000","last_became_active":"2026-02-13T02:22:16.007424+0000","last_became_peered":"2026-02-13T02:22:16.007424+0000","last_unstale":"2026-02-13T02:22:18.047121+0000","last_undegraded":"2026-02-13T02:22:18.047121+0000","last_fullsized":"2026-02-13T02:22:18.047121+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:10:41.888980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.020958+0000","last_change":"2026-02-13T02:22:17.017332+0000","last_active":"2026-02-13T02:22:18.020958+0000","last_peered":"2026-02-13T02:22:18.020958+0000","last_clean":"2026-02-13T02:22:18.020958+0000","last_became_active":"2026-02-13T02:22:17.017080+0000","last_became_peered":"2026-02-13T02:22:17.017080+0000","last_unstale":"2026-02-13T02:22:18.020958+0000","last_undegraded":"2026-02-13T02:22:18.020958+0000","last_fullsized":"2026-02-13T02:22:18.020958+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:10:16.349029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051313+0000","last_change":"2026-02-13T02:22:16.014204+0000","last_active":"2026-02-13T02:22:18.051313+0000","last_peered":"2026-02-13T02:22:18.051313+0000","last_clean":"2026-02-13T02:22:18.051313+0000","last_became_active":"2026-02-13T02:22:16.014114+0000","last_became_peered":"2026-02-13T02:22:16.014114+0000","last_unstale":"2026-02-13T02:22:18.051313+0000","last_undegraded":"2026-02-13T02:22:18.051313+0000","last_fullsized":"2026-02-13T02:22:18.051313+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:39:42.557823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.011965+0000","last_change":"2026-02-13T02:22:17.018644+0000","last_active":"2026-02-13T02:22:18.011965+0000","last_peered":"2026-02-13T02:22:18.011965+0000","last_clean":"2026-02-13T02:22:18.011965+0000","last_became_active":"2026-02-13T02:22:17.018511+0000","last_became_peered":"2026-02-13T02:22:17.018511+0000","last_unstale":"2026-02-13T02:22:18.011965+0000","last_undegraded":"2026-02-13T02:22:18.011965+0000","last_fullsized":"2026-02-13T02:22:18.011965+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:18:39.510344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051718+0000","last_change":"2026-02-13T02:22:16.013123+0000","last_active":"2026-02-13T02:22:18.051718+0000","last_peered":"2026-02-13T02:22:18.051718+0000","last_clean":"2026-02-13T02:22:18.051718+0000","last_became_active":"2026-02-13T02:22:16.012986+0000","last_became_peered":"2026-02-13T02:22:16.012986+0000","last_unstale":"2026-02-13T02:22:18.051718+0000","last_undegraded":"2026-02-13T02:22:18.051718+0000","last_fullsized":"2026-02-13T02:22:18.051718+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T10:09:29.447645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051655+0000","last_change":"2026-02-13T02:22:17.024718+0000","last_active":"2026-02-13T02:22:18.051655+0000","last_peered":"2026-02-13T02:22:18.051655+0000","last_clean":"2026-02-13T02:22:18.051655+0000","last_became_active":"2026-02-13T02:22:17.024333+0000","last_became_peered":"2026-02-13T02:22:17.024333+0000","last_unstale":"2026-02-13T02:22:18.051655+0000","last_undegraded":"2026-02-13T02:22:18.051655+0000","last_fullsized":"2026-02-13T02:22:18.051655+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:44:11.055118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051693+0000","last_change":"2026-02-13T02:22:16.015080+0000","last_active":"2026-02-13T02:22:18.051693+0000","last_peered":"2026-02-13T02:22:18.051693+0000","last_clean":"2026-02-13T02:22:18.051693+0000","last_became_active":"2026-02-13T02:22:16.014694+0000","last_became_peered":"2026-02-13T02:22:16.014694+0000","last_unstale":"2026-02-13T02:22:18.051693+0000","last_undegraded":"2026-02-13T02:22:18.051693+0000","last_fullsized":"2026-02-13T02:22:18.051693+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T06:02:28.163616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047603+0000","last_change":"2026-02-13T02:22:17.019516+0000","last_active":"2026-02-13T02:22:18.047603+0000","last_peered":"2026-02-13T02:22:18.047603+0000","last_clean":"2026-02-13T02:22:18.047603+0000","last_became_active":"2026-02-13T02:22:17.019234+0000","last_became_peered":"2026-02-13T02:22:17.019234+0000","last_unstale":"2026-02-13T02:22:18.047603+0000","last_undegraded":"2026-02-13T02:22:18.047603+0000","last_fullsized":"2026-02-13T02:22:18.047603+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T10:25:31.467398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"69'3","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051378+0000","last_change":"2026-02-13T02:22:16.013733+0000","last_active":"2026-02-13T02:22:18.051378+0000","last_peered":"2026-02-13T02:22:18.051378+0000","last_clean":"2026-02-13T02:22:18.051378+0000","last_became_active":"2026-02-13T02:22:16.013472+0000","last_became_peered":"2026-02-13T02:22:16.013472+0000","last_unstale":"2026-02-13T02:22:18.051378+0000","last_undegraded":"2026-02-13T02:22:18.051378+0000","last_fullsized":"2026-02-13T02:22:18.051378+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:59:36.928070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051362+0000","last_change":"2026-02-13T02:22:17.023907+0000","last_active":"2026-02-13T02:22:18.051362+0000","last_peered":"2026-02-13T02:22:18.051362+0000","last_clean":"2026-02-13T02:22:18.051362+0000","last_became_active":"2026-02-13T02:22:17.022420+0000","last_became_peered":"2026-02-13T02:22:17.022420+0000","last_unstale":"2026-02-13T02:22:18.051362+0000","last_undegraded":"2026-02-13T02:22:18.051362+0000","last_fullsized":"2026-02-13T02:22:18.051362+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:29:25.440291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"70'20","reported_seq":57,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:27.041983+0000","last_change":"2026-02-13T02:22:16.013888+0000","last_active":"2026-02-13T02:22:27.041983+0000","last_peered":"2026-02-13T02:22:27.041983+0000","last_clean":"2026-02-13T02:22:27.041983+0000","last_became_active":"2026-02-13T02:22:16.013652+0000","last_became_peered":"2026-02-13T02:22:16.013652+0000","last_unstale":"2026-02-13T02:22:27.041983+0000","last_undegraded":"2026-02-13T02:22:27.041983+0000","last_fullsized":"2026-02-13T02:22:27.041983+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:25:28.136946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52844,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":62,"num_write":20,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.025043+0000","last_change":"2026-02-13T02:22:17.023547+0000","last_active":"2026-02-13T02:22:18.025043+0000","last_peered":"2026-02-13T02:22:18.025043+0000","last_clean":"2026-02-13T02:22:18.025043+0000","last_became_active":"2026-02-13T02:22:17.023410+0000","last_became_peered":"2026-02-13T02:22:17.023410+0000","last_unstale":"2026-02-13T02:22:18.025043+0000","last_undegraded":"2026-02-13T02:22:18.025043+0000","last_fullsized":"2026-02-13T02:22:18.025043+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T04:35:03.522242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"69'2","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.507624+0000","last_change":"2026-02-13T02:22:16.014701+0000","last_active":"2026-02-13T02:22:18.507624+0000","last_peered":"2026-02-13T02:22:18.507624+0000","last_clean":"2026-02-13T02:22:18.507624+0000","last_became_active":"2026-02-13T02:22:16.014476+0000","last_became_peered":"2026-02-13T02:22:16.014476+0000","last_unstale":"2026-02-13T02:22:18.507624+0000","last_undegraded":"2026-02-13T02:22:18.507624+0000","last_fullsized":"2026-02-13T02:22:18.507624+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:25:47.865783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047309+0000","last_change":"2026-02-13T02:22:17.017924+0000","last_active":"2026-02-13T02:22:18.047309+0000","last_peered":"2026-02-13T02:22:18.047309+0000","last_clean":"2026-02-13T02:22:18.047309+0000","last_became_active":"2026-02-13T02:22:17.017666+0000","last_became_peered":"2026-02-13T02:22:17.017666+0000","last_unstale":"2026-02-13T02:22:18.047309+0000","last_undegraded":"2026-02-13T02:22:18.047309+0000","last_fullsized":"2026-02-13T02:22:18.047309+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:38:38.063114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051219+0000","last_change":"2026-02-13T02:22:16.014437+0000","last_active":"2026-02-13T02:22:18.051219+0000","last_peered":"2026-02-13T02:22:18.051219+0000","last_clean":"2026-02-13T02:22:18.051219+0000","last_became_active":"2026-02-13T02:22:16.014293+0000","last_became_peered":"2026-02-13T02:22:16.014293+0000","last_unstale":"2026-02-13T02:22:18.051219+0000","last_undegraded":"2026-02-13T02:22:18.051219+0000","last_fullsized":"2026-02-13T02:22:18.051219+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:08:58.340775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051558+0000","last_change":"2026-02-13T02:22:17.023621+0000","last_active":"2026-02-13T02:22:18.051558+0000","last_peered":"2026-02-13T02:22:18.051558+0000","last_clean":"2026-02-13T02:22:18.051558+0000","last_became_active":"2026-02-13T02:22:17.023343+0000","last_became_peered":"2026-02-13T02:22:17.023343+0000","last_unstale":"2026-02-13T02:22:18.051558+0000","last_undegraded":"2026-02-13T02:22:18.051558+0000","last_fullsized":"2026-02-13T02:22:18.051558+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T06:36:08.489377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.015962+0000","last_change":"2026-02-13T02:22:16.013825+0000","last_active":"2026-02-13T02:22:18.015962+0000","last_peered":"2026-02-13T02:22:18.015962+0000","last_clean":"2026-02-13T02:22:18.015962+0000","last_became_active":"2026-02-13T02:22:16.013704+0000","last_became_peered":"2026-02-13T02:22:16.013704+0000","last_unstale":"2026-02-13T02:22:18.015962+0000","last_undegraded":"2026-02-13T02:22:18.015962+0000","last_fullsized":"2026-02-13T02:22:18.015962+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T02:40:53.106423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051148+0000","last_change":"2026-02-13T02:22:17.022293+0000","last_active":"2026-02-13T02:22:18.051148+0000","last_peered":"2026-02-13T02:22:18.051148+0000","last_clean":"2026-02-13T02:22:18.051148+0000","last_became_active":"2026-02-13T02:22:17.022130+0000","last_became_peered":"2026-02-13T02:22:17.022130+0000","last_unstale":"2026-02-13T02:22:18.051148+0000","last_undegraded":"2026-02-13T02:22:18.051148+0000","last_fullsized":"2026-02-13T02:22:18.051148+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:33:03.123178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"69'8","reported_seq":24,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.024641+0000","last_change":"2026-02-13T02:22:16.013612+0000","last_active":"2026-02-13T02:22:18.024641+0000","last_peered":"2026-02-13T02:22:18.024641+0000","last_clean":"2026-02-13T02:22:18.024641+0000","last_became_active":"2026-02-13T02:22:16.013444+0000","last_became_peered":"2026-02-13T02:22:16.013444+0000","last_unstale":"2026-02-13T02:22:18.024641+0000","last_undegraded":"2026-02-13T02:22:18.024641+0000","last_fullsized":"2026-02-13T02:22:18.024641+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T04:59:32.263657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051261+0000","last_change":"2026-02-13T02:22:17.024692+0000","last_active":"2026-02-13T02:22:18.051261+0000","last_peered":"2026-02-13T02:22:18.051261+0000","last_clean":"2026-02-13T02:22:18.051261+0000","last_became_active":"2026-02-13T02:22:17.024543+0000","last_became_peered":"2026-02-13T02:22:17.024543+0000","last_unstale":"2026-02-13T02:22:18.051261+0000","last_undegraded":"2026-02-13T02:22:18.051261+0000","last_fullsized":"2026-02-13T02:22:18.051261+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:04:25.726793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"69'1","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.025073+0000","last_change":"2026-02-13T02:22:16.011678+0000","last_active":"2026-02-13T02:22:18.025073+0000","last_peered":"2026-02-13T02:22:18.025073+0000","last_clean":"2026-02-13T02:22:18.025073+0000","last_became_active":"2026-02-13T02:22:16.010914+0000","last_became_peered":"2026-02-13T02:22:16.010914+0000","last_unstale":"2026-02-13T02:22:18.025073+0000","last_undegraded":"2026-02-13T02:22:18.025073+0000","last_fullsized":"2026-02-13T02:22:18.025073+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:44:29.583923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016496+0000","last_change":"2026-02-13T02:22:17.023312+0000","last_active":"2026-02-13T02:22:18.016496+0000","last_peered":"2026-02-13T02:22:18.016496+0000","last_clean":"2026-02-13T02:22:18.016496+0000","last_became_active":"2026-02-13T02:22:17.023084+0000","last_became_peered":"2026-02-13T02:22:17.023084+0000","last_unstale":"2026-02-13T02:22:18.016496+0000","last_undegraded":"2026-02-13T02:22:18.016496+0000","last_fullsized":"2026-02-13T02:22:18.016496+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T10:27:31.848680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"69'2","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047389+0000","last_change":"2026-02-13T02:22:16.010281+0000","last_active":"2026-02-13T02:22:18.047389+0000","last_peered":"2026-02-13T02:22:18.047389+0000","last_clean":"2026-02-13T02:22:18.047389+0000","last_became_active":"2026-02-13T02:22:16.010075+0000","last_became_peered":"2026-02-13T02:22:16.010075+0000","last_unstale":"2026-02-13T02:22:18.047389+0000","last_undegraded":"2026-02-13T02:22:18.047389+0000","last_fullsized":"2026-02-13T02:22:18.047389+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:25:52.223102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051820+0000","last_change":"2026-02-13T02:22:17.023958+0000","last_active":"2026-02-13T02:22:18.051820+0000","last_peered":"2026-02-13T02:22:18.051820+0000","last_clean":"2026-02-13T02:22:18.051820+0000","last_became_active":"2026-02-13T02:22:17.023529+0000","last_became_peered":"2026-02-13T02:22:17.023529+0000","last_unstale":"2026-02-13T02:22:18.051820+0000","last_undegraded":"2026-02-13T02:22:18.051820+0000","last_fullsized":"2026-02-13T02:22:18.051820+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T12:45:26.937375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"69'1","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:19.304376+0000","last_change":"2026-02-13T02:22:16.013663+0000","last_active":"2026-02-13T02:22:19.304376+0000","last_peered":"2026-02-13T02:22:19.304376+0000","last_clean":"2026-02-13T02:22:19.304376+0000","last_became_active":"2026-02-13T02:22:16.013445+0000","last_became_peered":"2026-02-13T02:22:16.013445+0000","last_unstale":"2026-02-13T02:22:19.304376+0000","last_undegraded":"2026-02-13T02:22:19.304376+0000","last_fullsized":"2026-02-13T02:22:19.304376+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:02:58.149380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"70'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:19.481305+0000","last_change":"2026-02-13T02:22:17.021199+0000","last_active":"2026-02-13T02:22:19.481305+0000","last_peered":"2026-02-13T02:22:19.481305+0000","last_clean":"2026-02-13T02:22:19.481305+0000","last_became_active":"2026-02-13T02:22:17.021008+0000","last_became_peered":"2026-02-13T02:22:17.021008+0000","last_unstale":"2026-02-13T02:22:19.481305+0000","last_undegraded":"2026-02-13T02:22:19.481305+0000","last_fullsized":"2026-02-13T02:22:19.481305+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T09:21:08.010400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016275+0000","last_change":"2026-02-13T02:22:16.017721+0000","last_active":"2026-02-13T02:22:18.016275+0000","last_peered":"2026-02-13T02:22:18.016275+0000","last_clean":"2026-02-13T02:22:18.016275+0000","last_became_active":"2026-02-13T02:22:16.017552+0000","last_became_peered":"2026-02-13T02:22:16.017552+0000","last_unstale":"2026-02-13T02:22:18.016275+0000","last_undegraded":"2026-02-13T02:22:18.016275+0000","last_fullsized":"2026-02-13T02:22:18.016275+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:14:18.034709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051822+0000","last_change":"2026-02-13T02:22:17.025362+0000","last_active":"2026-02-13T02:22:18.051822+0000","last_peered":"2026-02-13T02:22:18.051822+0000","last_clean":"2026-02-13T02:22:18.051822+0000","last_became_active":"2026-02-13T02:22:17.024942+0000","last_became_peered":"2026-02-13T02:22:17.024942+0000","last_unstale":"2026-02-13T02:22:18.051822+0000","last_undegraded":"2026-02-13T02:22:18.051822+0000","last_fullsized":"2026-02-13T02:22:18.051822+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:09:37.454791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'152","reported_seq":339,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051251+0000","last_change":"2026-02-13T02:21:55.384973+0000","last_active":"2026-02-13T02:22:18.051251+0000","last_peered":"2026-02-13T02:22:18.051251+0000","last_clean":"2026-02-13T02:22:18.051251+0000","last_became_active":"2026-02-13T02:21:55.384610+0000","last_became_peered":"2026-02-13T02:21:55.384610+0000","last_unstale":"2026-02-13T02:22:18.051251+0000","last_undegraded":"2026-02-13T02:22:18.051251+0000","last_fullsized":"2026-02-13T02:22:18.051251+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:20:55.621197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:20:55.621197+0000","last_clean_scrub_stamp":"2026-02-13T02:20:55.621197+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:45:14.548172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":189,"num_write_kb":3716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016150+0000","last_change":"2026-02-13T02:22:16.012172+0000","last_active":"2026-02-13T02:22:18.016150+0000","last_peered":"2026-02-13T02:22:18.016150+0000","last_clean":"2026-02-13T02:22:18.016150+0000","last_became_active":"2026-02-13T02:22:16.011646+0000","last_became_peered":"2026-02-13T02:22:16.011646+0000","last_unstale":"2026-02-13T02:22:18.016150+0000","last_undegraded":"2026-02-13T02:22:18.016150+0000","last_fullsized":"2026-02-13T02:22:18.016150+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:05:01.007415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.021079+0000","last_change":"2026-02-13T02:22:17.020741+0000","last_active":"2026-02-13T02:22:18.021079+0000","last_peered":"2026-02-13T02:22:18.021079+0000","last_clean":"2026-02-13T02:22:18.021079+0000","last_became_active":"2026-02-13T02:22:17.020574+0000","last_became_peered":"2026-02-13T02:22:17.020574+0000","last_unstale":"2026-02-13T02:22:18.021079+0000","last_undegraded":"2026-02-13T02:22:18.021079+0000","last_fullsized":"2026-02-13T02:22:18.021079+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T06:28:11.279493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051246+0000","last_change":"2026-02-13T02:22:16.014565+0000","last_active":"2026-02-13T02:22:18.051246+0000","last_peered":"2026-02-13T02:22:18.051246+0000","last_clean":"2026-02-13T02:22:18.051246+0000","last_became_active":"2026-02-13T02:22:16.013815+0000","last_became_peered":"2026-02-13T02:22:16.013815+0000","last_unstale":"2026-02-13T02:22:18.051246+0000","last_undegraded":"2026-02-13T02:22:18.051246+0000","last_fullsized":"2026-02-13T02:22:18.051246+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T04:34:31.290868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047168+0000","last_change":"2026-02-13T02:22:17.013289+0000","last_active":"2026-02-13T02:22:18.047168+0000","last_peered":"2026-02-13T02:22:18.047168+0000","last_clean":"2026-02-13T02:22:18.047168+0000","last_became_active":"2026-02-13T02:22:17.013160+0000","last_became_peered":"2026-02-13T02:22:17.013160+0000","last_unstale":"2026-02-13T02:22:18.047168+0000","last_undegraded":"2026-02-13T02:22:18.047168+0000","last_fullsized":"2026-02-13T02:22:18.047168+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:19:06.942113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047197+0000","last_change":"2026-02-13T02:22:16.007420+0000","last_active":"2026-02-13T02:22:18.047197+0000","last_peered":"2026-02-13T02:22:18.047197+0000","last_clean":"2026-02-13T02:22:18.047197+0000","last_became_active":"2026-02-13T02:22:16.007114+0000","last_became_peered":"2026-02-13T02:22:16.007114+0000","last_unstale":"2026-02-13T02:22:18.047197+0000","last_undegraded":"2026-02-13T02:22:18.047197+0000","last_fullsized":"2026-02-13T02:22:18.047197+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:54:40.155679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051284+0000","last_change":"2026-02-13T02:22:17.021679+0000","last_active":"2026-02-13T02:22:18.051284+0000","last_peered":"2026-02-13T02:22:18.051284+0000","last_clean":"2026-02-13T02:22:18.051284+0000","last_became_active":"2026-02-13T02:22:17.021041+0000","last_became_peered":"2026-02-13T02:22:17.021041+0000","last_unstale":"2026-02-13T02:22:18.051284+0000","last_undegraded":"2026-02-13T02:22:18.051284+0000","last_fullsized":"2026-02-13T02:22:18.051284+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T04:37:08.611062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051417+0000","last_change":"2026-02-13T02:22:16.016749+0000","last_active":"2026-02-13T02:22:18.051417+0000","last_peered":"2026-02-13T02:22:18.051417+0000","last_clean":"2026-02-13T02:22:18.051417+0000","last_became_active":"2026-02-13T02:22:16.016620+0000","last_became_peered":"2026-02-13T02:22:16.016620+0000","last_unstale":"2026-02-13T02:22:18.051417+0000","last_undegraded":"2026-02-13T02:22:18.051417+0000","last_fullsized":"2026-02-13T02:22:18.051417+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:29:12.370221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051646+0000","last_change":"2026-02-13T02:22:17.025708+0000","last_active":"2026-02-13T02:22:18.051646+0000","last_peered":"2026-02-13T02:22:18.051646+0000","last_clean":"2026-02-13T02:22:18.051646+0000","last_became_active":"2026-02-13T02:22:17.025240+0000","last_became_peered":"2026-02-13T02:22:17.025240+0000","last_unstale":"2026-02-13T02:22:18.051646+0000","last_undegraded":"2026-02-13T02:22:18.051646+0000","last_fullsized":"2026-02-13T02:22:18.051646+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T07:04:02.085370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.011935+0000","last_change":"2026-02-13T02:22:16.009592+0000","last_active":"2026-02-13T02:22:18.011935+0000","last_peered":"2026-02-13T02:22:18.011935+0000","last_clean":"2026-02-13T02:22:18.011935+0000","last_became_active":"2026-02-13T02:22:16.009465+0000","last_became_peered":"2026-02-13T02:22:16.009465+0000","last_unstale":"2026-02-13T02:22:18.011935+0000","last_undegraded":"2026-02-13T02:22:18.011935+0000","last_fullsized":"2026-02-13T02:22:18.011935+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T02:33:29.133373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"70'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051885+0000","last_change":"2026-02-13T02:22:17.023729+0000","last_active":"2026-02-13T02:22:18.051885+0000","last_peered":"2026-02-13T02:22:18.051885+0000","last_clean":"2026-02-13T02:22:18.051885+0000","last_became_active":"2026-02-13T02:22:17.022007+0000","last_became_peered":"2026-02-13T02:22:17.022007+0000","last_unstale":"2026-02-13T02:22:18.051885+0000","last_undegraded":"2026-02-13T02:22:18.051885+0000","last_fullsized":"2026-02-13T02:22:18.051885+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T10:34:43.688805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.025077+0000","last_change":"2026-02-13T02:22:16.012023+0000","last_active":"2026-02-13T02:22:18.025077+0000","last_peered":"2026-02-13T02:22:18.025077+0000","last_clean":"2026-02-13T02:22:18.025077+0000","last_became_active":"2026-02-13T02:22:16.011680+0000","last_became_peered":"2026-02-13T02:22:16.011680+0000","last_unstale":"2026-02-13T02:22:18.025077+0000","last_undegraded":"2026-02-13T02:22:18.025077+0000","last_fullsized":"2026-02-13T02:22:18.025077+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T02:28:59.078471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047450+0000","last_change":"2026-02-13T02:22:17.015877+0000","last_active":"2026-02-13T02:22:18.047450+0000","last_peered":"2026-02-13T02:22:18.047450+0000","last_clean":"2026-02-13T02:22:18.047450+0000","last_became_active":"2026-02-13T02:22:17.015701+0000","last_became_peered":"2026-02-13T02:22:17.015701+0000","last_unstale":"2026-02-13T02:22:18.047450+0000","last_undegraded":"2026-02-13T02:22:18.047450+0000","last_fullsized":"2026-02-13T02:22:18.047450+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:35:45.903754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"70'6","reported_seq":22,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:28.021286+0000","last_change":"2026-02-13T02:22:16.008749+0000","last_active":"2026-02-13T02:22:28.021286+0000","last_peered":"2026-02-13T02:22:28.021286+0000","last_clean":"2026-02-13T02:22:28.021286+0000","last_became_active":"2026-02-13T02:22:16.008597+0000","last_became_peered":"2026-02-13T02:22:16.008597+0000","last_unstale":"2026-02-13T02:22:28.021286+0000","last_undegraded":"2026-02-13T02:22:28.021286+0000","last_fullsized":"2026-02-13T02:22:28.021286+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T10:18:25.963625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.021323+0000","last_change":"2026-02-13T02:22:17.017797+0000","last_active":"2026-02-13T02:22:18.021323+0000","last_peered":"2026-02-13T02:22:18.021323+0000","last_clean":"2026-02-13T02:22:18.021323+0000","last_became_active":"2026-02-13T02:22:17.017174+0000","last_became_peered":"2026-02-13T02:22:17.017174+0000","last_unstale":"2026-02-13T02:22:18.021323+0000","last_undegraded":"2026-02-13T02:22:18.021323+0000","last_fullsized":"2026-02-13T02:22:18.021323+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:39:41.102807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"69'2","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016292+0000","last_change":"2026-02-13T02:22:16.017646+0000","last_active":"2026-02-13T02:22:18.016292+0000","last_peered":"2026-02-13T02:22:18.016292+0000","last_clean":"2026-02-13T02:22:18.016292+0000","last_became_active":"2026-02-13T02:22:16.017529+0000","last_became_peered":"2026-02-13T02:22:16.017529+0000","last_unstale":"2026-02-13T02:22:18.016292+0000","last_undegraded":"2026-02-13T02:22:18.016292+0000","last_fullsized":"2026-02-13T02:22:18.016292+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T12:34:48.977200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047594+0000","last_change":"2026-02-13T02:22:17.019503+0000","last_active":"2026-02-13T02:22:18.047594+0000","last_peered":"2026-02-13T02:22:18.047594+0000","last_clean":"2026-02-13T02:22:18.047594+0000","last_became_active":"2026-02-13T02:22:17.019259+0000","last_became_peered":"2026-02-13T02:22:17.019259+0000","last_unstale":"2026-02-13T02:22:18.047594+0000","last_undegraded":"2026-02-13T02:22:18.047594+0000","last_fullsized":"2026-02-13T02:22:18.047594+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:19:49.339245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.050971+0000","last_change":"2026-02-13T02:22:16.016248+0000","last_active":"2026-02-13T02:22:18.050971+0000","last_peered":"2026-02-13T02:22:18.050971+0000","last_clean":"2026-02-13T02:22:18.050971+0000","last_became_active":"2026-02-13T02:22:16.016016+0000","last_became_peered":"2026-02-13T02:22:16.016016+0000","last_unstale":"2026-02-13T02:22:18.050971+0000","last_undegraded":"2026-02-13T02:22:18.050971+0000","last_fullsized":"2026-02-13T02:22:18.050971+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:39:11.300311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.024643+0000","last_change":"2026-02-13T02:22:17.023656+0000","last_active":"2026-02-13T02:22:18.024643+0000","last_peered":"2026-02-13T02:22:18.024643+0000","last_clean":"2026-02-13T02:22:18.024643+0000","last_became_active":"2026-02-13T02:22:17.023524+0000","last_became_peered":"2026-02-13T02:22:17.023524+0000","last_unstale":"2026-02-13T02:22:18.024643+0000","last_undegraded":"2026-02-13T02:22:18.024643+0000","last_fullsized":"2026-02-13T02:22:18.024643+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:33:34.390446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.020934+0000","last_change":"2026-02-13T02:22:16.010000+0000","last_active":"2026-02-13T02:22:18.020934+0000","last_peered":"2026-02-13T02:22:18.020934+0000","last_clean":"2026-02-13T02:22:18.020934+0000","last_became_active":"2026-02-13T02:22:16.009736+0000","last_became_peered":"2026-02-13T02:22:16.009736+0000","last_unstale":"2026-02-13T02:22:18.020934+0000","last_undegraded":"2026-02-13T02:22:18.020934+0000","last_fullsized":"2026-02-13T02:22:18.020934+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:16:46.363188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.016302+0000","last_change":"2026-02-13T02:22:17.022375+0000","last_active":"2026-02-13T02:22:18.016302+0000","last_peered":"2026-02-13T02:22:18.016302+0000","last_clean":"2026-02-13T02:22:18.016302+0000","last_became_active":"2026-02-13T02:22:17.022236+0000","last_became_peered":"2026-02-13T02:22:17.022236+0000","last_unstale":"2026-02-13T02:22:18.016302+0000","last_undegraded":"2026-02-13T02:22:18.016302+0000","last_fullsized":"2026-02-13T02:22:18.016302+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:32:39.302116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.015763+0000","last_change":"2026-02-13T02:22:16.015222+0000","last_active":"2026-02-13T02:22:18.015763+0000","last_peered":"2026-02-13T02:22:18.015763+0000","last_clean":"2026-02-13T02:22:18.015763+0000","last_became_active":"2026-02-13T02:22:16.014947+0000","last_became_peered":"2026-02-13T02:22:16.014947+0000","last_unstale":"2026-02-13T02:22:18.015763+0000","last_undegraded":"2026-02-13T02:22:18.015763+0000","last_fullsized":"2026-02-13T02:22:18.015763+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T14:01:02.677585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.021118+0000","last_change":"2026-02-13T02:22:17.021948+0000","last_active":"2026-02-13T02:22:18.021118+0000","last_peered":"2026-02-13T02:22:18.021118+0000","last_clean":"2026-02-13T02:22:18.021118+0000","last_became_active":"2026-02-13T02:22:17.021764+0000","last_became_peered":"2026-02-13T02:22:17.021764+0000","last_unstale":"2026-02-13T02:22:18.021118+0000","last_undegraded":"2026-02-13T02:22:18.021118+0000","last_fullsized":"2026-02-13T02:22:18.021118+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:39:50.788814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"69'2","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.024813+0000","last_change":"2026-02-13T02:22:16.014740+0000","last_active":"2026-02-13T02:22:18.024813+0000","last_peered":"2026-02-13T02:22:18.024813+0000","last_clean":"2026-02-13T02:22:18.024813+0000","last_became_active":"2026-02-13T02:22:16.014578+0000","last_became_peered":"2026-02-13T02:22:16.014578+0000","last_unstale":"2026-02-13T02:22:18.024813+0000","last_undegraded":"2026-02-13T02:22:18.024813+0000","last_fullsized":"2026-02-13T02:22:18.024813+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T11:09:10.619351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051553+0000","last_change":"2026-02-13T02:22:17.025211+0000","last_active":"2026-02-13T02:22:18.051553+0000","last_peered":"2026-02-13T02:22:18.051553+0000","last_clean":"2026-02-13T02:22:18.051553+0000","last_became_active":"2026-02-13T02:22:17.025015+0000","last_became_peered":"2026-02-13T02:22:17.025015+0000","last_unstale":"2026-02-13T02:22:18.051553+0000","last_undegraded":"2026-02-13T02:22:18.051553+0000","last_fullsized":"2026-02-13T02:22:18.051553+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T04:28:08.456016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051394+0000","last_change":"2026-02-13T02:22:17.022434+0000","last_active":"2026-02-13T02:22:18.051394+0000","last_peered":"2026-02-13T02:22:18.051394+0000","last_clean":"2026-02-13T02:22:18.051394+0000","last_became_active":"2026-02-13T02:22:17.022160+0000","last_became_peered":"2026-02-13T02:22:17.022160+0000","last_unstale":"2026-02-13T02:22:18.051394+0000","last_undegraded":"2026-02-13T02:22:18.051394+0000","last_fullsized":"2026-02-13T02:22:18.051394+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T12:27:17.994699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"69'2","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.024901+0000","last_change":"2026-02-13T02:22:16.013867+0000","last_active":"2026-02-13T02:22:18.024901+0000","last_peered":"2026-02-13T02:22:18.024901+0000","last_clean":"2026-02-13T02:22:18.024901+0000","last_became_active":"2026-02-13T02:22:16.013689+0000","last_became_peered":"2026-02-13T02:22:16.013689+0000","last_unstale":"2026-02-13T02:22:18.024901+0000","last_undegraded":"2026-02-13T02:22:18.024901+0000","last_fullsized":"2026-02-13T02:22:18.024901+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T04:46:39.476015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.051182+0000","last_change":"2026-02-13T02:22:17.022407+0000","last_active":"2026-02-13T02:22:18.051182+0000","last_peered":"2026-02-13T02:22:18.051182+0000","last_clean":"2026-02-13T02:22:18.051182+0000","last_became_active":"2026-02-13T02:22:17.021354+0000","last_became_peered":"2026-02-13T02:22:17.021354+0000","last_unstale":"2026-02-13T02:22:18.051182+0000","last_undegraded":"2026-02-13T02:22:18.051182+0000","last_fullsized":"2026-02-13T02:22:18.051182+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T05:35:18.694829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"69'2","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.025017+0000","last_change":"2026-02-13T02:22:16.015538+0000","last_active":"2026-02-13T02:22:18.025017+0000","last_peered":"2026-02-13T02:22:18.025017+0000","last_clean":"2026-02-13T02:22:18.025017+0000","last_became_active":"2026-02-13T02:22:16.015441+0000","last_became_peered":"2026-02-13T02:22:16.015441+0000","last_unstale":"2026-02-13T02:22:18.025017+0000","last_undegraded":"2026-02-13T02:22:18.025017+0000","last_fullsized":"2026-02-13T02:22:18.025017+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T02:34:39.849666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.047574+0000","last_change":"2026-02-13T02:22:17.018483+0000","last_active":"2026-02-13T02:22:18.047574+0000","last_peered":"2026-02-13T02:22:18.047574+0000","last_clean":"2026-02-13T02:22:18.047574+0000","last_became_active":"2026-02-13T02:22:17.018318+0000","last_became_peered":"2026-02-13T02:22:17.018318+0000","last_unstale":"2026-02-13T02:22:18.047574+0000","last_undegraded":"2026-02-13T02:22:18.047574+0000","last_fullsized":"2026-02-13T02:22:18.047574+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T10:22:49.603621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"69'1","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.021277+0000","last_change":"2026-02-13T02:22:16.008795+0000","last_active":"2026-02-13T02:22:18.021277+0000","last_peered":"2026-02-13T02:22:18.021277+0000","last_clean":"2026-02-13T02:22:18.021277+0000","last_became_active":"2026-02-13T02:22:16.008621+0000","last_became_peered":"2026-02-13T02:22:16.008621+0000","last_unstale":"2026-02-13T02:22:18.021277+0000","last_undegraded":"2026-02-13T02:22:18.021277+0000","last_fullsized":"2026-02-13T02:22:18.021277+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T03:43:45.861077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:18.021240+0000","last_change":"2026-02-13T02:22:17.019826+0000","last_active":"2026-02-13T02:22:18.021240+0000","last_peered":"2026-02-13T02:22:18.021240+0000","last_clean":"2026-02-13T02:22:18.021240+0000","last_became_active":"2026-02-13T02:22:17.019429+0000","last_became_peered":"2026-02-13T02:22:17.019429+0000","last_unstale":"2026-02-13T02:22:18.021240+0000","last_undegraded":"2026-02-13T02:22:18.021240+0000","last_fullsized":"2026-02-13T02:22:18.021240+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:16.001121+0000","last_clean_scrub_stamp":"2026-02-13T02:22:16.001121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T13:52:59.597088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"69'4","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2026-02-13T02:22:19.205618+0000","last_change":"2026-02-13T02:22:16.011002+0000","last_active":"2026-02-13T02:22:19.205618+0000","last_peered":"2026-02-13T02:22:19.205618+0000","last_clean":"2026-02-13T02:22:19.205618+0000","last_became_active":"2026-02-13T02:22:16.010892+0000","last_became_peered":"2026-02-13T02:22:16.010892+0000","last_unstale":"2026-02-13T02:22:19.205618+0000","last_undegraded":"2026-02-13T02:22:19.205618+0000","last_fullsized":"2026-02-13T02:22:19.205618+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-13T02:22:14.999954+0000","last_clean_scrub_stamp":"2026-02-13T02:22:14.999954+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-14T08:57:22.333777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":604,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":53664,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":27,"num_read_kb":68,"num_write":69,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":160992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":69480,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":189,"num_write_kb":3716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":264161,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":65,"seq":279172874248,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27696,"kb_used_data":732,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":733975504,"statfs":{"total":751619276800,"available":751590916096,"internally_reserved":0,"allocated":749568,"data_stored":364081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8466,"internal_metadata":27582190},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":58,"seq":249108103177,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27820,"kb_used_data":856,"kb_used_omap":18,"kb_used_meta":26925,"kb_avail":733975380,"statfs":{"total":751619276800,"available":751590789120,"internally_reserved":0,"allocated":876544,"data_stored":1007604,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":19356,"internal_metadata":27571300},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":54,"seq":231928233996,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27700,"kb_used_data":736,"kb_used_omap":13,"kb_used_meta":26930,"kb_avail":733975500,"statfs":{"total":751619276800,"available":751590912000,"internally_reserved":0,"allocated":753664,"data_stored":364685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13327,"internal_metadata":27577329},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":47,"seq":201863462924,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27704,"kb_used_data":740,"kb_used_omap":21,"kb_used_meta":26922,"kb_avail":733975496,"statfs":{"total":751619276800,"available":751590907904,"internally_reserved":0,"allocated":757760,"data_stored":364707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22372,"internal_metadata":27568284},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":40,"seq":171798691854,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27752,"kb_used_data":788,"kb_used_omap":23,"kb_used_meta":26920,"kb_avail":733975448,"statfs":{"total":751619276800,"available":751590858752,"internally_reserved":0,"allocated":806912,"data_stored":955170,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":24500,"internal_metadata":27566156},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":37,"seq":158913789968,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27700,"kb_used_data":736,"kb_used_omap":23,"kb_used_meta":26920,"kb_avail":733975500,"statfs":{"total":751619276800,"available":751590912000,"internally_reserved":0,"allocated":753664,"data_stored":364198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24460,"internal_metadata":27566196},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":30,"seq":128849018898,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27760,"kb_used_data":796,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":733975440,"statfs":{"total":751619276800,"available":751590850560,"internally_reserved":0,"allocated":815104,"data_stored":954748,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":29636,"internal_metadata":27561020},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":26,"seq":111669149715,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27700,"kb_used_data":736,"kb_used_omap":23,"kb_used_meta":26920,"kb_avail":733975500,"statfs":{"total":751619276800,"available":751590912000,"internally_reserved":0,"allocated":753664,"data_stored":364103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24268,"internal_metadata":27566388},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":19,"seq":81604378645,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27712,"kb_used_data":748,"kb_used_omap":41,"kb_used_meta":26902,"kb_avail":733975488,"statfs":{"total":751619276800,"available":751590899712,"internally_reserved":0,"allocated":765952,"data_stored":364419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42463,"internal_metadata":27548193},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27760,"kb_used_data":796,"kb_used_omap":48,"kb_used_meta":26895,"kb_avail":733975440,"statfs":{"total":751619276800,"available":751590850560,"internally_reserved":0,"allocated":815104,"data_stored":416959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49734,"internal_metadata":27540922},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672983,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27752,"kb_used_data":788,"kb_used_omap":38,"kb_used_meta":26905,"kb_avail":733975448,"statfs":{"total":751619276800,"available":751590858752,"internally_reserved":0,"allocated":806912,"data_stored":411609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39406,"internal_metadata":27551250},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":7,"seq":30064771097,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":27700,"kb_used_data":736,"kb_used_omap":40,"kb_used_meta":26903,"kb_avail":733975500,"statfs":{"total":751619276800,"available":751590912000,"internally_reserved":0,"allocated":753664,"data_stored":364171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41083,"internal_metadata":27549573},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":36563,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":36563,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38207,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20097,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":52844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":705,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":52878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13826,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2307,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1258,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":366,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1586,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":656,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5747,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1185,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":53038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17757,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3990,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-13T02:22:31.350 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2026-02-13T02:22:31.351 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-13T02:22:31.351 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2026-02-13T02:22:31.351 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:22:31.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:31 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/213258991' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2026-02-13T02:22:31.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:31 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/953242410' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2026-02-13T02:22:31.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:31 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/4036026419' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2026-02-13T02:22:31.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:31 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2385519176' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2026-02-13T02:22:31.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:31 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2319221594' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2026-02-13T02:22:31.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:31 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/158182147' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2026-02-13T02:22:31.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:31 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3563207069' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2026-02-13T02:22:31.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:31 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2697967832' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2026-02-13T02:22:31.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:31 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/213258991' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2026-02-13T02:22:31.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:31 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/953242410' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2026-02-13T02:22:31.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:31 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/4036026419' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2026-02-13T02:22:31.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:31 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2385519176' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2026-02-13T02:22:31.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:31 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2319221594' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2026-02-13T02:22:31.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:31 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/158182147' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2026-02-13T02:22:31.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:31 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3563207069' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2026-02-13T02:22:31.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:31 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2697967832' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2026-02-13T02:22:31.501 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:31.627 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.624+0000 7fc249b2f6c0 1 -- 10.20.193.27:0/2197172165 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc24410f230 msgr2=0x7fc24410f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:31.627 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.624+0000 7fc249b2f6c0 1 --2- 10.20.193.27:0/2197172165 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc24410f230 0x7fc24410f600 secure :-1 s=READY pgs=334 gs=1 cs=0 l=1 c_cookie=3653109151026238779 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc2300099a0 tx=0x7fc23002ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:31.627 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.624+0000 7fc249b2f6c0 1 -- 10.20.193.27:0/2197172165 shutdown_connections 2026-02-13T02:22:31.627 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.624+0000 7fc249b2f6c0 1 --2- 10.20.193.27:0/2197172165 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc244115440 0x7fc244175de0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:31.628 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.624+0000 7fc249b2f6c0 1 --2- 10.20.193.27:0/2197172165 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc24410fc20 0x7fc244114c60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:31.628 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.624+0000 7fc249b2f6c0 1 --2- 10.20.193.27:0/2197172165 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc24410f230 0x7fc24410f600 unknown :-1 s=CLOSED pgs=334 gs=1 cs=0 l=1 c_cookie=3653109151026238779 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:31.628 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.624+0000 7fc249b2f6c0 1 -- 10.20.193.27:0/2197172165 >> 10.20.193.27:0/2197172165 conn(0x7fc2440ecb30 msgr2=0x7fc24410cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:31.628 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.624+0000 7fc249b2f6c0 1 -- 10.20.193.27:0/2197172165 shutdown_connections 2026-02-13T02:22:31.628 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.624+0000 7fc249b2f6c0 1 -- 10.20.193.27:0/2197172165 wait complete. 2026-02-13T02:22:31.628 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.624+0000 7fc249b2f6c0 1 Processor -- start 2026-02-13T02:22:31.628 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.624+0000 7fc249b2f6c0 1 -- start start 2026-02-13T02:22:31.628 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.625+0000 7fc249b2f6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc24410f230 0x7fc24411d280 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:31.628 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.625+0000 7fc249b2f6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc244115440 0x7fc24411d7c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:31.628 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.625+0000 7fc249b2f6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc244122b80 0x7fc244123020 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:31.628 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.625+0000 7fc249b2f6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc244123560 con 0x7fc24410f230 2026-02-13T02:22:31.628 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.625+0000 7fc249b2f6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc2441236d0 con 0x7fc244115440 2026-02-13T02:22:31.629 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.625+0000 7fc249b2f6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc244123890 con 0x7fc244122b80 2026-02-13T02:22:31.629 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.625+0000 7fc2437fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc24410f230 0x7fc24411d280 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:31.629 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.625+0000 7fc2437fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc24410f230 0x7fc24411d280 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:54032/0 (socket says 10.20.193.27:54032) 2026-02-13T02:22:31.629 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.625+0000 7fc2437fe6c0 1 -- 10.20.193.27:0/2403360146 learned_addr learned my addr 10.20.193.27:0/2403360146 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:31.629 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.625+0000 7fc242ffd6c0 1 --2- 10.20.193.27:0/2403360146 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc244115440 0x7fc24411d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:31.629 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.625+0000 7fc243fff6c0 1 --2- 10.20.193.27:0/2403360146 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc244122b80 0x7fc244123020 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:31.629 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.625+0000 7fc2437fe6c0 1 -- 10.20.193.27:0/2403360146 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc244122b80 msgr2=0x7fc244123020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:31.629 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.625+0000 7fc2437fe6c0 1 --2- 10.20.193.27:0/2403360146 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc244122b80 0x7fc244123020 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:31.629 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.625+0000 7fc2437fe6c0 1 -- 10.20.193.27:0/2403360146 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc244115440 msgr2=0x7fc24411d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:31.629 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.625+0000 7fc2437fe6c0 1 --2- 10.20.193.27:0/2403360146 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc244115440 0x7fc24411d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:31.629 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.625+0000 7fc2437fe6c0 1 -- 10.20.193.27:0/2403360146 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc230009590 con 0x7fc24410f230 2026-02-13T02:22:31.629 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.625+0000 7fc243fff6c0 1 --2- 10.20.193.27:0/2403360146 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc244122b80 0x7fc244123020 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:22:31.630 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.625+0000 7fc2437fe6c0 1 --2- 10.20.193.27:0/2403360146 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc24410f230 0x7fc24411d280 secure :-1 s=READY pgs=335 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc23002f230 tx=0x7fc23002fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:31.630 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.625+0000 7fc240ff96c0 1 -- 10.20.193.27:0/2403360146 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc230002ba0 con 0x7fc24410f230 2026-02-13T02:22:31.630 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.625+0000 7fc249b2f6c0 1 -- 10.20.193.27:0/2403360146 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc244163d10 con 0x7fc24410f230 2026-02-13T02:22:31.630 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.625+0000 7fc249b2f6c0 1 -- 10.20.193.27:0/2403360146 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2441641e0 con 0x7fc24410f230 2026-02-13T02:22:31.630 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.625+0000 7fc240ff96c0 1 -- 10.20.193.27:0/2403360146 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc230007680 con 0x7fc24410f230 2026-02-13T02:22:31.630 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.625+0000 7fc240ff96c0 1 -- 10.20.193.27:0/2403360146 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc230037480 con 0x7fc24410f230 2026-02-13T02:22:31.630 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.625+0000 7fc249b2f6c0 1 -- 10.20.193.27:0/2403360146 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc244110500 con 0x7fc24410f230 2026-02-13T02:22:31.631 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.627+0000 7fc240ff96c0 1 -- 10.20.193.27:0/2403360146 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7fc230002d00 con 0x7fc24410f230 2026-02-13T02:22:31.631 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.628+0000 7fc240ff96c0 1 --2- 10.20.193.27:0/2403360146 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc2180b5710 0x7fc2180d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:31.631 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.628+0000 7fc242ffd6c0 1 --2- 10.20.193.27:0/2403360146 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc2180b5710 0x7fc2180d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:31.631 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.628+0000 7fc240ff96c0 1 -- 10.20.193.27:0/2403360146 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7fc230107d40 con 0x7fc24410f230 2026-02-13T02:22:31.632 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.628+0000 7fc240ff96c0 1 -- 10.20.193.27:0/2403360146 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fc2300ca7f0 con 0x7fc24410f230 2026-02-13T02:22:31.632 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.628+0000 7fc242ffd6c0 1 --2- 10.20.193.27:0/2403360146 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc2180b5710 0x7fc2180d5bc0 secure :-1 s=READY pgs=231 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc234005980 tx=0x7fc234005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:31.748 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.747+0000 7fc249b2f6c0 1 -- 10.20.193.27:0/2403360146 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc24410f600 con 0x7fc24410f230 2026-02-13T02:22:31.748 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.747+0000 7fc240ff96c0 1 -- 10.20.193.27:0/2403360146 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7fc2300cd910 con 0x7fc24410f230 2026-02-13T02:22:31.748 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:22:31.749 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:22:31.750 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.748+0000 7fc249b2f6c0 1 -- 10.20.193.27:0/2403360146 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc2180b5710 msgr2=0x7fc2180d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:31.750 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.748+0000 7fc249b2f6c0 1 --2- 10.20.193.27:0/2403360146 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc2180b5710 0x7fc2180d5bc0 secure :-1 s=READY pgs=231 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc234005980 tx=0x7fc234005910 comp rx=0 tx=0).stop 2026-02-13T02:22:31.750 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.748+0000 7fc249b2f6c0 1 -- 10.20.193.27:0/2403360146 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc24410f230 msgr2=0x7fc24411d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:31.750 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.748+0000 7fc249b2f6c0 1 --2- 10.20.193.27:0/2403360146 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc24410f230 0x7fc24411d280 secure :-1 s=READY pgs=335 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc23002f230 tx=0x7fc23002fd40 comp rx=0 tx=0).stop 2026-02-13T02:22:31.750 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.748+0000 7fc249b2f6c0 1 -- 10.20.193.27:0/2403360146 shutdown_connections 2026-02-13T02:22:31.750 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.748+0000 7fc249b2f6c0 1 --2- 10.20.193.27:0/2403360146 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc2180b5710 0x7fc2180d5bc0 unknown :-1 s=CLOSED pgs=231 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:31.750 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.748+0000 7fc249b2f6c0 1 --2- 10.20.193.27:0/2403360146 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc244122b80 0x7fc244123020 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:31.750 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.748+0000 7fc249b2f6c0 1 --2- 10.20.193.27:0/2403360146 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc244115440 0x7fc24411d7c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:31.750 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.748+0000 7fc249b2f6c0 1 --2- 10.20.193.27:0/2403360146 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc24410f230 0x7fc24411d280 unknown :-1 s=CLOSED pgs=335 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:31.750 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.748+0000 7fc249b2f6c0 1 -- 10.20.193.27:0/2403360146 >> 10.20.193.27:0/2403360146 conn(0x7fc2440ecb30 msgr2=0x7fc24410cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:31.751 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.748+0000 7fc249b2f6c0 1 -- 10.20.193.27:0/2403360146 shutdown_connections 2026-02-13T02:22:31.751 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:31.748+0000 7fc249b2f6c0 1 -- 10.20.193.27:0/2403360146 wait complete. 2026-02-13T02:22:32.394 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:32 trial028 ceph-mon[26701]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-13T02:22:32.395 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:32 trial028 ceph-mon[26701]: pgmap v191: 97 pgs: 97 active+clean; 629 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 5.7 KiB/s rd, 6.2 KiB/s wr, 7 op/s 2026-02-13T02:22:32.395 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:32 trial028 ceph-mon[26701]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-13T02:22:32.395 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:32 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2403360146' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:22:32.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:32 trial027 ceph-mon[24567]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-13T02:22:32.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:32 trial027 ceph-mon[24567]: pgmap v191: 97 pgs: 97 active+clean; 629 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 5.7 KiB/s rd, 6.2 KiB/s wr, 7 op/s 2026-02-13T02:22:32.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:32 trial027 ceph-mon[24567]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-13T02:22:32.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:32 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2403360146' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:22:32.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:32 trial107 ceph-mon[27422]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-13T02:22:32.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:32 trial107 ceph-mon[27422]: pgmap v191: 97 pgs: 97 active+clean; 629 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 5.7 KiB/s rd, 6.2 KiB/s wr, 7 op/s 2026-02-13T02:22:32.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:32 trial107 ceph-mon[27422]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-13T02:22:32.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:32 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2403360146' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:22:34.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:34 trial027 ceph-mon[24567]: pgmap v192: 97 pgs: 97 active+clean; 645 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 6.0 KiB/s rd, 7.5 KiB/s wr, 7 op/s 2026-02-13T02:22:34.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:34 trial028 ceph-mon[26701]: pgmap v192: 97 pgs: 97 active+clean; 645 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 6.0 KiB/s rd, 7.5 KiB/s wr, 7 op/s 2026-02-13T02:22:34.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:34 trial107 ceph-mon[27422]: pgmap v192: 97 pgs: 97 active+clean; 645 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 6.0 KiB/s rd, 7.5 KiB/s wr, 7 op/s 2026-02-13T02:22:34.789 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:22:34.908 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:35.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.021+0000 7f8a424e46c0 1 -- 10.20.193.27:0/210602752 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8a3c115440 msgr2=0x7f8a3c175de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:35.026 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.021+0000 7f8a424e46c0 1 --2- 10.20.193.27:0/210602752 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8a3c115440 0x7f8a3c175de0 secure :-1 s=READY pgs=336 gs=1 cs=0 l=1 c_cookie=792153757817739679 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8a3000aa70 tx=0x7f8a3002fde0 comp rx=0 tx=0).stop 2026-02-13T02:22:35.026 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.021+0000 7f8a424e46c0 1 -- 10.20.193.27:0/210602752 shutdown_connections 2026-02-13T02:22:35.026 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.021+0000 7f8a424e46c0 1 --2- 10.20.193.27:0/210602752 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8a3c115440 0x7f8a3c175de0 unknown :-1 s=CLOSED pgs=336 gs=1 cs=0 l=1 c_cookie=792153757817739679 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:35.026 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.021+0000 7f8a424e46c0 1 --2- 10.20.193.27:0/210602752 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8a3c10fc20 0x7f8a3c114c60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:35.026 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.021+0000 7f8a424e46c0 1 --2- 10.20.193.27:0/210602752 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8a3c10f230 0x7f8a3c10f600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:35.026 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.021+0000 7f8a424e46c0 1 -- 10.20.193.27:0/210602752 >> 10.20.193.27:0/210602752 conn(0x7f8a3c0ecb30 msgr2=0x7f8a3c10cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:35.026 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.021+0000 7f8a424e46c0 1 -- 10.20.193.27:0/210602752 shutdown_connections 2026-02-13T02:22:35.026 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.021+0000 7f8a424e46c0 1 -- 10.20.193.27:0/210602752 wait complete. 2026-02-13T02:22:35.027 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.021+0000 7f8a424e46c0 1 Processor -- start 2026-02-13T02:22:35.027 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.021+0000 7f8a424e46c0 1 -- start start 2026-02-13T02:22:35.027 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.022+0000 7f8a424e46c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8a3c10f230 0x7f8a3c11d200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:35.027 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.022+0000 7f8a424e46c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8a3c115440 0x7f8a3c11d740 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:35.027 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.022+0000 7f8a424e46c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8a3c122b00 0x7f8a3c122fa0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:35.027 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.022+0000 7f8a424e46c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c123510 con 0x7f8a3c10f230 2026-02-13T02:22:35.027 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.022+0000 7f8a424e46c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c123680 con 0x7f8a3c122b00 2026-02-13T02:22:35.027 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.022+0000 7f8a424e46c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c123840 con 0x7f8a3c115440 2026-02-13T02:22:35.027 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.022+0000 7f8a3bfff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8a3c10f230 0x7f8a3c11d200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:35.027 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.022+0000 7f8a3bfff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8a3c10f230 0x7f8a3c11d200 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:54046/0 (socket says 10.20.193.27:54046) 2026-02-13T02:22:35.027 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.022+0000 7f8a3bfff6c0 1 -- 10.20.193.27:0/4275389605 learned_addr learned my addr 10.20.193.27:0/4275389605 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:35.027 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.022+0000 7f8a3b7fe6c0 1 --2- 10.20.193.27:0/4275389605 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8a3c115440 0x7f8a3c11d740 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:35.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.022+0000 7f8a3bfff6c0 1 -- 10.20.193.27:0/4275389605 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8a3c115440 msgr2=0x7f8a3c11d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:35.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.022+0000 7f8a3bfff6c0 1 --2- 10.20.193.27:0/4275389605 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8a3c115440 0x7f8a3c11d740 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:35.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.022+0000 7f8a3bfff6c0 1 -- 10.20.193.27:0/4275389605 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8a3c122b00 msgr2=0x7f8a3c122fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:35.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.022+0000 7f8a3bfff6c0 1 --2- 10.20.193.27:0/4275389605 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8a3c122b00 0x7f8a3c122fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:35.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.022+0000 7f8a3bfff6c0 1 -- 10.20.193.27:0/4275389605 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3000a660 con 0x7f8a3c10f230 2026-02-13T02:22:35.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.022+0000 7f8a3bfff6c0 1 --2- 10.20.193.27:0/4275389605 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8a3c10f230 0x7f8a3c11d200 secure :-1 s=READY pgs=337 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8a28005dc0 tx=0x7f8a2800bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:35.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.022+0000 7f8a397fa6c0 1 -- 10.20.193.27:0/4275389605 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8a28004250 con 0x7f8a3c10f230 2026-02-13T02:22:35.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.022+0000 7f8a397fa6c0 1 -- 10.20.193.27:0/4275389605 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f8a28009070 con 0x7f8a3c10f230 2026-02-13T02:22:35.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.022+0000 7f8a397fa6c0 1 -- 10.20.193.27:0/4275389605 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8a2800cd70 con 0x7f8a3c10f230 2026-02-13T02:22:35.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.022+0000 7f8a424e46c0 1 -- 10.20.193.27:0/4275389605 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a3c163cf0 con 0x7f8a3c10f230 2026-02-13T02:22:35.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.022+0000 7f8a424e46c0 1 -- 10.20.193.27:0/4275389605 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a3c164180 con 0x7f8a3c10f230 2026-02-13T02:22:35.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.022+0000 7f8a424e46c0 1 -- 10.20.193.27:0/4275389605 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a3c110500 con 0x7f8a3c10f230 2026-02-13T02:22:35.029 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.023+0000 7f8a397fa6c0 1 -- 10.20.193.27:0/4275389605 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f8a280028c0 con 0x7f8a3c10f230 2026-02-13T02:22:35.029 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.024+0000 7f8a397fa6c0 1 --2- 10.20.193.27:0/4275389605 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8a100b5710 0x7f8a100d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:35.030 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.024+0000 7f8a3b7fe6c0 1 --2- 10.20.193.27:0/4275389605 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8a100b5710 0x7f8a100d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:35.030 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.024+0000 7f8a397fa6c0 1 -- 10.20.193.27:0/4275389605 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f8a280e3b20 con 0x7f8a3c10f230 2026-02-13T02:22:35.030 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.024+0000 7f8a3b7fe6c0 1 --2- 10.20.193.27:0/4275389605 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8a100b5710 0x7f8a100d5bc0 secure :-1 s=READY pgs=232 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8a3c123b90 tx=0x7f8a2c0094b0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:35.030 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.024+0000 7f8a397fa6c0 1 -- 10.20.193.27:0/4275389605 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f8a280a6540 con 0x7f8a3c10f230 2026-02-13T02:22:35.141 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.139+0000 7f8a424e46c0 1 -- 10.20.193.27:0/4275389605 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8a3c10fc20 con 0x7f8a3c10f230 2026-02-13T02:22:35.141 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.139+0000 7f8a397fa6c0 1 -- 10.20.193.27:0/4275389605 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f8a280a96f0 con 0x7f8a3c10f230 2026-02-13T02:22:35.141 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:22:35.142 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:22:35.143 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.140+0000 7f8a424e46c0 1 -- 10.20.193.27:0/4275389605 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8a100b5710 msgr2=0x7f8a100d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:35.143 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.140+0000 7f8a424e46c0 1 --2- 10.20.193.27:0/4275389605 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8a100b5710 0x7f8a100d5bc0 secure :-1 s=READY pgs=232 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8a3c123b90 tx=0x7f8a2c0094b0 comp rx=0 tx=0).stop 2026-02-13T02:22:35.143 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.140+0000 7f8a424e46c0 1 -- 10.20.193.27:0/4275389605 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8a3c10f230 msgr2=0x7f8a3c11d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:35.143 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.140+0000 7f8a424e46c0 1 --2- 10.20.193.27:0/4275389605 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8a3c10f230 0x7f8a3c11d200 secure :-1 s=READY pgs=337 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8a28005dc0 tx=0x7f8a2800bbb0 comp rx=0 tx=0).stop 2026-02-13T02:22:35.143 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.140+0000 7f8a424e46c0 1 -- 10.20.193.27:0/4275389605 shutdown_connections 2026-02-13T02:22:35.143 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.140+0000 7f8a424e46c0 1 --2- 10.20.193.27:0/4275389605 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8a100b5710 0x7f8a100d5bc0 unknown :-1 s=CLOSED pgs=232 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:35.143 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.140+0000 7f8a424e46c0 1 --2- 10.20.193.27:0/4275389605 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8a3c122b00 0x7f8a3c122fa0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:35.143 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.140+0000 7f8a424e46c0 1 --2- 10.20.193.27:0/4275389605 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8a3c115440 0x7f8a3c11d740 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:35.143 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.140+0000 7f8a424e46c0 1 --2- 10.20.193.27:0/4275389605 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8a3c10f230 0x7f8a3c11d200 unknown :-1 s=CLOSED pgs=337 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:35.143 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.140+0000 7f8a424e46c0 1 -- 10.20.193.27:0/4275389605 >> 10.20.193.27:0/4275389605 conn(0x7f8a3c0ecb30 msgr2=0x7f8a3c10cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:35.144 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.140+0000 7f8a424e46c0 1 -- 10.20.193.27:0/4275389605 shutdown_connections 2026-02-13T02:22:35.144 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:35.140+0000 7f8a424e46c0 1 -- 10.20.193.27:0/4275389605 wait complete. 2026-02-13T02:22:36.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:36 trial027 ceph-mon[24567]: pgmap v193: 97 pgs: 97 active+clean; 645 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 5.4 KiB/s rd, 6.4 KiB/s wr, 4 op/s 2026-02-13T02:22:36.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:36 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/4275389605' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:22:36.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:36 trial028 ceph-mon[26701]: pgmap v193: 97 pgs: 97 active+clean; 645 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 5.4 KiB/s rd, 6.4 KiB/s wr, 4 op/s 2026-02-13T02:22:36.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:36 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/4275389605' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:22:36.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:36 trial107 ceph-mon[27422]: pgmap v193: 97 pgs: 97 active+clean; 645 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 5.4 KiB/s rd, 6.4 KiB/s wr, 4 op/s 2026-02-13T02:22:36.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:36 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/4275389605' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:22:38.185 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:22:38.304 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:38.326 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:38 trial027 ceph-mon[24567]: pgmap v194: 97 pgs: 97 active+clean; 645 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 4 op/s 2026-02-13T02:22:38.427 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.422+0000 7fb37ee5b6c0 1 -- 10.20.193.27:0/1082592557 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb37810fbf0 msgr2=0x7fb378114c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:38.427 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.422+0000 7fb37ee5b6c0 1 --2- 10.20.193.27:0/1082592557 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb37810fbf0 0x7fb378114c30 secure :-1 s=READY pgs=338 gs=1 cs=0 l=1 c_cookie=9955297127961659054 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb3600099a0 tx=0x7fb36002ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:38.427 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.422+0000 7fb37ee5b6c0 1 -- 10.20.193.27:0/1082592557 shutdown_connections 2026-02-13T02:22:38.427 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.422+0000 7fb37ee5b6c0 1 --2- 10.20.193.27:0/1082592557 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb378115410 0x7fb378175db0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:38.428 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.422+0000 7fb37ee5b6c0 1 --2- 10.20.193.27:0/1082592557 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb37810fbf0 0x7fb378114c30 unknown :-1 s=CLOSED pgs=338 gs=1 cs=0 l=1 c_cookie=9955297127961659054 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:38.428 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.422+0000 7fb37ee5b6c0 1 --2- 10.20.193.27:0/1082592557 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb37810f200 0x7fb37810f5d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:38.428 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.422+0000 7fb37ee5b6c0 1 -- 10.20.193.27:0/1082592557 >> 10.20.193.27:0/1082592557 conn(0x7fb3780ecb00 msgr2=0x7fb37810cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:38.428 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.422+0000 7fb37ee5b6c0 1 -- 10.20.193.27:0/1082592557 shutdown_connections 2026-02-13T02:22:38.428 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.422+0000 7fb37ee5b6c0 1 -- 10.20.193.27:0/1082592557 wait complete. 2026-02-13T02:22:38.428 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.423+0000 7fb37ee5b6c0 1 Processor -- start 2026-02-13T02:22:38.428 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.423+0000 7fb37ee5b6c0 1 -- start start 2026-02-13T02:22:38.428 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.423+0000 7fb37ee5b6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb37810f200 0x7fb3780f4ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:38.428 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.423+0000 7fb37ee5b6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb378115410 0x7fb378175cb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:38.428 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.423+0000 7fb37ee5b6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb3780f5310 0x7fb3780fa5e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:38.428 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.423+0000 7fb37ee5b6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb3780f5a00 con 0x7fb37810f200 2026-02-13T02:22:38.429 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.423+0000 7fb37ee5b6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb3780fab20 con 0x7fb3780f5310 2026-02-13T02:22:38.429 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.423+0000 7fb37ee5b6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb3780face0 con 0x7fb378115410 2026-02-13T02:22:38.429 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.423+0000 7fb37cbcc6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb37810f200 0x7fb3780f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:38.429 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.423+0000 7fb37cbcc6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb37810f200 0x7fb3780f4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:57292/0 (socket says 10.20.193.27:57292) 2026-02-13T02:22:38.429 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.423+0000 7fb37cbcc6c0 1 -- 10.20.193.27:0/701624217 learned_addr learned my addr 10.20.193.27:0/701624217 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:38.429 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.423+0000 7fb37d3cd6c0 1 --2- 10.20.193.27:0/701624217 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb3780f5310 0x7fb3780fa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:38.429 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.423+0000 7fb36ffff6c0 1 --2- 10.20.193.27:0/701624217 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb378115410 0x7fb378175cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:38.429 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.423+0000 7fb37cbcc6c0 1 -- 10.20.193.27:0/701624217 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb378115410 msgr2=0x7fb378175cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:38.429 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.423+0000 7fb37cbcc6c0 1 --2- 10.20.193.27:0/701624217 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb378115410 0x7fb378175cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:38.429 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.423+0000 7fb37cbcc6c0 1 -- 10.20.193.27:0/701624217 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb3780f5310 msgr2=0x7fb3780fa5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:38.429 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.423+0000 7fb37cbcc6c0 1 --2- 10.20.193.27:0/701624217 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb3780f5310 0x7fb3780fa5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:38.430 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.423+0000 7fb37cbcc6c0 1 -- 10.20.193.27:0/701624217 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb360009590 con 0x7fb37810f200 2026-02-13T02:22:38.430 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.423+0000 7fb37cbcc6c0 1 --2- 10.20.193.27:0/701624217 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb37810f200 0x7fb3780f4ce0 secure :-1 s=READY pgs=339 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb368009490 tx=0x7fb36800bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:38.430 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.423+0000 7fb36dffb6c0 1 -- 10.20.193.27:0/701624217 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb368004a80 con 0x7fb37810f200 2026-02-13T02:22:38.430 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.423+0000 7fb36dffb6c0 1 -- 10.20.193.27:0/701624217 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb368009030 con 0x7fb37810f200 2026-02-13T02:22:38.430 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.423+0000 7fb37ee5b6c0 1 -- 10.20.193.27:0/701624217 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3780fafc0 con 0x7fb37810f200 2026-02-13T02:22:38.430 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.423+0000 7fb36dffb6c0 1 -- 10.20.193.27:0/701624217 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb36800ce30 con 0x7fb37810f200 2026-02-13T02:22:38.430 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.423+0000 7fb37ee5b6c0 1 -- 10.20.193.27:0/701624217 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3780fb750 con 0x7fb37810f200 2026-02-13T02:22:38.430 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.424+0000 7fb37ee5b6c0 1 -- 10.20.193.27:0/701624217 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3781104d0 con 0x7fb37810f200 2026-02-13T02:22:38.431 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.425+0000 7fb36dffb6c0 1 -- 10.20.193.27:0/701624217 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7fb368004060 con 0x7fb37810f200 2026-02-13T02:22:38.431 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.426+0000 7fb36dffb6c0 1 --2- 10.20.193.27:0/701624217 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb3540b5710 0x7fb3540d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:38.431 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.426+0000 7fb36ffff6c0 1 --2- 10.20.193.27:0/701624217 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb3540b5710 0x7fb3540d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:38.432 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.426+0000 7fb36dffb6c0 1 -- 10.20.193.27:0/701624217 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7fb3680e3ac0 con 0x7fb37810f200 2026-02-13T02:22:38.432 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.426+0000 7fb36ffff6c0 1 --2- 10.20.193.27:0/701624217 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb3540b5710 0x7fb3540d5bc0 secure :-1 s=READY pgs=233 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb360005b40 tx=0x7fb3600092a0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:38.432 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.426+0000 7fb36dffb6c0 1 -- 10.20.193.27:0/701624217 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fb3680a9f40 con 0x7fb37810f200 2026-02-13T02:22:38.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:38 trial028 ceph-mon[26701]: pgmap v194: 97 pgs: 97 active+clean; 645 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 4 op/s 2026-02-13T02:22:38.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:38 trial107 ceph-mon[27422]: pgmap v194: 97 pgs: 97 active+clean; 645 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 4 op/s 2026-02-13T02:22:38.542 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.540+0000 7fb37ee5b6c0 1 -- 10.20.193.27:0/701624217 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb3780f5e30 con 0x7fb37810f200 2026-02-13T02:22:38.543 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.540+0000 7fb36dffb6c0 1 -- 10.20.193.27:0/701624217 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7fb3680a9690 con 0x7fb37810f200 2026-02-13T02:22:38.543 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:22:38.543 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:22:38.544 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.541+0000 7fb37ee5b6c0 1 -- 10.20.193.27:0/701624217 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb3540b5710 msgr2=0x7fb3540d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:38.544 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.541+0000 7fb37ee5b6c0 1 --2- 10.20.193.27:0/701624217 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb3540b5710 0x7fb3540d5bc0 secure :-1 s=READY pgs=233 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb360005b40 tx=0x7fb3600092a0 comp rx=0 tx=0).stop 2026-02-13T02:22:38.544 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.541+0000 7fb37ee5b6c0 1 -- 10.20.193.27:0/701624217 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb37810f200 msgr2=0x7fb3780f4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:38.544 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.541+0000 7fb37ee5b6c0 1 --2- 10.20.193.27:0/701624217 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb37810f200 0x7fb3780f4ce0 secure :-1 s=READY pgs=339 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb368009490 tx=0x7fb36800bca0 comp rx=0 tx=0).stop 2026-02-13T02:22:38.544 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.541+0000 7fb37ee5b6c0 1 -- 10.20.193.27:0/701624217 shutdown_connections 2026-02-13T02:22:38.545 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.541+0000 7fb37ee5b6c0 1 --2- 10.20.193.27:0/701624217 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb3540b5710 0x7fb3540d5bc0 unknown :-1 s=CLOSED pgs=233 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:38.545 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.541+0000 7fb37ee5b6c0 1 --2- 10.20.193.27:0/701624217 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb3780f5310 0x7fb3780fa5e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:38.545 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.541+0000 7fb37ee5b6c0 1 --2- 10.20.193.27:0/701624217 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb378115410 0x7fb378175cb0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:38.545 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.541+0000 7fb37ee5b6c0 1 --2- 10.20.193.27:0/701624217 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb37810f200 0x7fb3780f4ce0 unknown :-1 s=CLOSED pgs=339 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:38.545 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.541+0000 7fb37ee5b6c0 1 -- 10.20.193.27:0/701624217 >> 10.20.193.27:0/701624217 conn(0x7fb3780ecb00 msgr2=0x7fb37810d900 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:38.545 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.541+0000 7fb37ee5b6c0 1 -- 10.20.193.27:0/701624217 shutdown_connections 2026-02-13T02:22:38.545 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:38.541+0000 7fb37ee5b6c0 1 -- 10.20.193.27:0/701624217 wait complete. 2026-02-13T02:22:39.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:39 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/701624217' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:22:39.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:39 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/701624217' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:22:39.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:39 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/701624217' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:22:40.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:40 trial027 ceph-mon[24567]: pgmap v195: 97 pgs: 97 active+clean; 645 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2026-02-13T02:22:40.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:40 trial028 ceph-mon[26701]: pgmap v195: 97 pgs: 97 active+clean; 645 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2026-02-13T02:22:40.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:40 trial107 ceph-mon[27422]: pgmap v195: 97 pgs: 97 active+clean; 645 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2026-02-13T02:22:41.600 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:22:41.719 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:41.831 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.826+0000 7f49434636c0 1 -- 10.20.193.27:0/292286657 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f493c115410 msgr2=0x7f493c175db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:41.831 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.826+0000 7f49434636c0 1 --2- 10.20.193.27:0/292286657 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f493c115410 0x7f493c175db0 secure :-1 s=READY pgs=340 gs=1 cs=0 l=1 c_cookie=13013946076346812769 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f493800aa70 tx=0x7f493802fde0 comp rx=0 tx=0).stop 2026-02-13T02:22:41.831 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.826+0000 7f49434636c0 1 -- 10.20.193.27:0/292286657 shutdown_connections 2026-02-13T02:22:41.831 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.826+0000 7f49434636c0 1 --2- 10.20.193.27:0/292286657 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f493c115410 0x7f493c175db0 unknown :-1 s=CLOSED pgs=340 gs=1 cs=0 l=1 c_cookie=13013946076346812769 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:41.831 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.826+0000 7f49434636c0 1 --2- 10.20.193.27:0/292286657 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f493c10fbf0 0x7f493c114c30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:41.831 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.826+0000 7f49434636c0 1 --2- 10.20.193.27:0/292286657 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f493c10f200 0x7f493c10f5d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:41.831 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.826+0000 7f49434636c0 1 -- 10.20.193.27:0/292286657 >> 10.20.193.27:0/292286657 conn(0x7f493c0ecb00 msgr2=0x7f493c10cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:41.831 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.826+0000 7f49434636c0 1 -- 10.20.193.27:0/292286657 shutdown_connections 2026-02-13T02:22:41.832 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.826+0000 7f49434636c0 1 -- 10.20.193.27:0/292286657 wait complete. 2026-02-13T02:22:41.832 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.827+0000 7f49434636c0 1 Processor -- start 2026-02-13T02:22:41.832 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.827+0000 7f49434636c0 1 -- start start 2026-02-13T02:22:41.832 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.827+0000 7f49434636c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f493c10f200 0x7f493c0f4bd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:41.832 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.827+0000 7f49434636c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f493c115410 0x7f493c175cb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:41.832 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.827+0000 7f49434636c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f493c0f5200 0x7f493c0fa4d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:41.832 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.827+0000 7f49434636c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f493c0f58f0 con 0x7f493c115410 2026-02-13T02:22:41.832 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.827+0000 7f49434636c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f493c0faa10 con 0x7f493c10f200 2026-02-13T02:22:41.832 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.827+0000 7f49434636c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f493c0fabd0 con 0x7f493c0f5200 2026-02-13T02:22:41.832 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.827+0000 7f49409d36c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f493c115410 0x7f493c175cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:41.832 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.827+0000 7f49409d36c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f493c115410 0x7f493c175cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:57320/0 (socket says 10.20.193.27:57320) 2026-02-13T02:22:41.833 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.827+0000 7f49409d36c0 1 -- 10.20.193.27:0/1842041799 learned_addr learned my addr 10.20.193.27:0/1842041799 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:41.833 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.827+0000 7f49411d46c0 1 --2- 10.20.193.27:0/1842041799 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f493c10f200 0x7f493c0f4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:41.833 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.827+0000 7f49419d56c0 1 --2- 10.20.193.27:0/1842041799 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f493c0f5200 0x7f493c0fa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:41.833 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.827+0000 7f49409d36c0 1 -- 10.20.193.27:0/1842041799 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f493c0f5200 msgr2=0x7f493c0fa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:41.833 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.827+0000 7f49409d36c0 1 --2- 10.20.193.27:0/1842041799 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f493c0f5200 0x7f493c0fa4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:41.833 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.827+0000 7f49409d36c0 1 -- 10.20.193.27:0/1842041799 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f493c10f200 msgr2=0x7f493c0f4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:41.833 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.827+0000 7f49409d36c0 1 --2- 10.20.193.27:0/1842041799 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f493c10f200 0x7f493c0f4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:41.833 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.827+0000 7f49409d36c0 1 -- 10.20.193.27:0/1842041799 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f493800a660 con 0x7f493c115410 2026-02-13T02:22:41.833 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.827+0000 7f49419d56c0 1 --2- 10.20.193.27:0/1842041799 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f493c0f5200 0x7f493c0fa4d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:22:41.833 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.827+0000 7f49409d36c0 1 --2- 10.20.193.27:0/1842041799 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f493c115410 0x7f493c175cb0 secure :-1 s=READY pgs=341 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4930005dc0 tx=0x7f493000bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:41.833 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.827+0000 7f492a7fc6c0 1 -- 10.20.193.27:0/1842041799 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4930004250 con 0x7f493c115410 2026-02-13T02:22:41.833 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.827+0000 7f492a7fc6c0 1 -- 10.20.193.27:0/1842041799 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4930009070 con 0x7f493c115410 2026-02-13T02:22:41.834 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.827+0000 7f492a7fc6c0 1 -- 10.20.193.27:0/1842041799 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f493000cde0 con 0x7f493c115410 2026-02-13T02:22:41.834 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.827+0000 7f49434636c0 1 -- 10.20.193.27:0/1842041799 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f493c0faeb0 con 0x7f493c115410 2026-02-13T02:22:41.834 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.827+0000 7f49434636c0 1 -- 10.20.193.27:0/1842041799 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f493c0fb640 con 0x7f493c115410 2026-02-13T02:22:41.834 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.828+0000 7f49434636c0 1 -- 10.20.193.27:0/1842041799 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f493c1104d0 con 0x7f493c115410 2026-02-13T02:22:41.835 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.829+0000 7f492a7fc6c0 1 -- 10.20.193.27:0/1842041799 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f49300028c0 con 0x7f493c115410 2026-02-13T02:22:41.835 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.830+0000 7f492a7fc6c0 1 --2- 10.20.193.27:0/1842041799 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f49180b5710 0x7f49180d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:41.835 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.830+0000 7f49411d46c0 1 --2- 10.20.193.27:0/1842041799 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f49180b5710 0x7f49180d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:41.835 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.830+0000 7f492a7fc6c0 1 -- 10.20.193.27:0/1842041799 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f49300e3a30 con 0x7f493c115410 2026-02-13T02:22:41.835 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.830+0000 7f49411d46c0 1 --2- 10.20.193.27:0/1842041799 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f49180b5710 0x7f49180d5bc0 secure :-1 s=READY pgs=234 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f492c005980 tx=0x7f492c005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:41.835 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.831+0000 7f492a7fc6c0 1 -- 10.20.193.27:0/1842041799 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f49300a6450 con 0x7f493c115410 2026-02-13T02:22:41.952 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.949+0000 7f49434636c0 1 -- 10.20.193.27:0/1842041799 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f493c0f5d20 con 0x7f493c115410 2026-02-13T02:22:41.952 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.949+0000 7f492a7fc6c0 1 -- 10.20.193.27:0/1842041799 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f49300a9600 con 0x7f493c115410 2026-02-13T02:22:41.952 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:22:41.952 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:22:41.954 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.951+0000 7f49434636c0 1 -- 10.20.193.27:0/1842041799 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f49180b5710 msgr2=0x7f49180d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:41.954 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.951+0000 7f49434636c0 1 --2- 10.20.193.27:0/1842041799 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f49180b5710 0x7f49180d5bc0 secure :-1 s=READY pgs=234 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f492c005980 tx=0x7f492c005910 comp rx=0 tx=0).stop 2026-02-13T02:22:41.954 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.951+0000 7f49434636c0 1 -- 10.20.193.27:0/1842041799 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f493c115410 msgr2=0x7f493c175cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:41.954 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.951+0000 7f49434636c0 1 --2- 10.20.193.27:0/1842041799 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f493c115410 0x7f493c175cb0 secure :-1 s=READY pgs=341 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4930005dc0 tx=0x7f493000bbb0 comp rx=0 tx=0).stop 2026-02-13T02:22:41.955 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.952+0000 7f49434636c0 1 -- 10.20.193.27:0/1842041799 shutdown_connections 2026-02-13T02:22:41.955 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.952+0000 7f49434636c0 1 --2- 10.20.193.27:0/1842041799 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f49180b5710 0x7f49180d5bc0 unknown :-1 s=CLOSED pgs=234 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:41.955 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.952+0000 7f49434636c0 1 --2- 10.20.193.27:0/1842041799 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f493c0f5200 0x7f493c0fa4d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:41.955 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.952+0000 7f49434636c0 1 --2- 10.20.193.27:0/1842041799 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f493c115410 0x7f493c175cb0 unknown :-1 s=CLOSED pgs=341 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:41.955 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.952+0000 7f49434636c0 1 --2- 10.20.193.27:0/1842041799 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f493c10f200 0x7f493c0f4bd0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:41.955 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.952+0000 7f49434636c0 1 -- 10.20.193.27:0/1842041799 >> 10.20.193.27:0/1842041799 conn(0x7f493c0ecb00 msgr2=0x7f493c10d800 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:41.955 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.952+0000 7f49434636c0 1 -- 10.20.193.27:0/1842041799 shutdown_connections 2026-02-13T02:22:41.955 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:41.952+0000 7f49434636c0 1 -- 10.20.193.27:0/1842041799 wait complete. 2026-02-13T02:22:42.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:42 trial027 ceph-mon[24567]: pgmap v196: 97 pgs: 97 active+clean; 645 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2026-02-13T02:22:42.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:42 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1842041799' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:22:42.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:42 trial028 ceph-mon[26701]: pgmap v196: 97 pgs: 97 active+clean; 645 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2026-02-13T02:22:42.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:42 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1842041799' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:22:42.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:42 trial107 ceph-mon[27422]: pgmap v196: 97 pgs: 97 active+clean; 645 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2026-02-13T02:22:42.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:42 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1842041799' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:22:44.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:44 trial027 ceph-mon[24567]: pgmap v197: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 2.6 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2026-02-13T02:22:44.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:44 trial028 ceph-mon[26701]: pgmap v197: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 2.6 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2026-02-13T02:22:44.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:44 trial107 ceph-mon[27422]: pgmap v197: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 2.6 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2026-02-13T02:22:44.997 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:22:45.117 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:45.233 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.229+0000 7f89214a36c0 1 -- 10.20.193.27:0/1822989597 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f891c10f200 msgr2=0x7f891c10f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:45.233 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.229+0000 7f89214a36c0 1 --2- 10.20.193.27:0/1822989597 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f891c10f200 0x7f891c10f5d0 secure :-1 s=READY pgs=342 gs=1 cs=0 l=1 c_cookie=9744877455034730215 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f89100099a0 tx=0x7f891002ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:45.233 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.229+0000 7f89214a36c0 1 -- 10.20.193.27:0/1822989597 shutdown_connections 2026-02-13T02:22:45.233 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.229+0000 7f89214a36c0 1 --2- 10.20.193.27:0/1822989597 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f891c115410 0x7f891c175db0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:45.233 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.229+0000 7f89214a36c0 1 --2- 10.20.193.27:0/1822989597 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f891c10fbf0 0x7f891c114c30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:45.233 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.229+0000 7f89214a36c0 1 --2- 10.20.193.27:0/1822989597 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f891c10f200 0x7f891c10f5d0 unknown :-1 s=CLOSED pgs=342 gs=1 cs=0 l=1 c_cookie=9744877455034730215 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:45.234 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.229+0000 7f89214a36c0 1 -- 10.20.193.27:0/1822989597 >> 10.20.193.27:0/1822989597 conn(0x7f891c0ecb00 msgr2=0x7f891c10cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:45.234 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.229+0000 7f89214a36c0 1 -- 10.20.193.27:0/1822989597 shutdown_connections 2026-02-13T02:22:45.234 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.229+0000 7f89214a36c0 1 -- 10.20.193.27:0/1822989597 wait complete. 2026-02-13T02:22:45.234 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.229+0000 7f89214a36c0 1 Processor -- start 2026-02-13T02:22:45.234 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.229+0000 7f89214a36c0 1 -- start start 2026-02-13T02:22:45.234 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.229+0000 7f89214a36c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f891c10f200 0x7f891c11d260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:45.234 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.229+0000 7f89214a36c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f891c115410 0x7f891c11d7a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:45.234 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.229+0000 7f89214a36c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f891c122b60 0x7f891c123000 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:45.234 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.229+0000 7f89214a36c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f891c123540 con 0x7f891c10f200 2026-02-13T02:22:45.234 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.229+0000 7f89214a36c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f891c1236b0 con 0x7f891c122b60 2026-02-13T02:22:45.234 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.229+0000 7f89214a36c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f891c123870 con 0x7f891c115410 2026-02-13T02:22:45.235 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.230+0000 7f891b7fe6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f891c122b60 0x7f891c123000 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:45.235 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.230+0000 7f891a7fc6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f891c115410 0x7f891c11d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:45.235 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.230+0000 7f891a7fc6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f891c115410 0x7f891c11d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.107:3300/0 says I am v2:10.20.193.27:57448/0 (socket says 10.20.193.27:57448) 2026-02-13T02:22:45.235 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.230+0000 7f891b7fe6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f891c122b60 0x7f891c123000 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.27:34980/0 (socket says 10.20.193.27:34980) 2026-02-13T02:22:45.235 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.230+0000 7f891a7fc6c0 1 -- 10.20.193.27:0/474388304 learned_addr learned my addr 10.20.193.27:0/474388304 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:45.235 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.230+0000 7f891affd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f891c10f200 0x7f891c11d260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:45.235 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.230+0000 7f891affd6c0 1 -- 10.20.193.27:0/474388304 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f891c115410 msgr2=0x7f891c11d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:45.235 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.230+0000 7f891affd6c0 1 --2- 10.20.193.27:0/474388304 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f891c115410 0x7f891c11d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:45.235 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.230+0000 7f891affd6c0 1 -- 10.20.193.27:0/474388304 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f891c122b60 msgr2=0x7f891c123000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:45.235 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.230+0000 7f891affd6c0 1 --2- 10.20.193.27:0/474388304 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f891c122b60 0x7f891c123000 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:45.235 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.230+0000 7f891affd6c0 1 -- 10.20.193.27:0/474388304 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8910009590 con 0x7f891c10f200 2026-02-13T02:22:45.235 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.230+0000 7f891affd6c0 1 --2- 10.20.193.27:0/474388304 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f891c10f200 0x7f891c11d260 secure :-1 s=READY pgs=343 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f891002f230 tx=0x7f891002fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:45.236 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.230+0000 7f88fbfff6c0 1 -- 10.20.193.27:0/474388304 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8910002ae0 con 0x7f891c10f200 2026-02-13T02:22:45.236 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.230+0000 7f88fbfff6c0 1 -- 10.20.193.27:0/474388304 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f8910002c40 con 0x7f891c10f200 2026-02-13T02:22:45.236 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.230+0000 7f89214a36c0 1 -- 10.20.193.27:0/474388304 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f891c163cf0 con 0x7f891c10f200 2026-02-13T02:22:45.236 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.230+0000 7f88fbfff6c0 1 -- 10.20.193.27:0/474388304 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f89100362f0 con 0x7f891c10f200 2026-02-13T02:22:45.236 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.230+0000 7f89214a36c0 1 -- 10.20.193.27:0/474388304 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f891c1641c0 con 0x7f891c10f200 2026-02-13T02:22:45.236 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.231+0000 7f89214a36c0 1 -- 10.20.193.27:0/474388304 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f891c1104d0 con 0x7f891c10f200 2026-02-13T02:22:45.237 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.232+0000 7f88fbfff6c0 1 -- 10.20.193.27:0/474388304 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f89100077a0 con 0x7f891c10f200 2026-02-13T02:22:45.237 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.232+0000 7f88fbfff6c0 1 --2- 10.20.193.27:0/474388304 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f88f00b5710 0x7f88f00d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:45.237 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.232+0000 7f891a7fc6c0 1 --2- 10.20.193.27:0/474388304 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f88f00b5710 0x7f88f00d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:45.237 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.232+0000 7f88fbfff6c0 1 -- 10.20.193.27:0/474388304 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f8910107bb0 con 0x7f891c10f200 2026-02-13T02:22:45.237 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.233+0000 7f891a7fc6c0 1 --2- 10.20.193.27:0/474388304 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f88f00b5710 0x7f88f00d5bc0 secure :-1 s=READY pgs=235 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f891c123bf0 tx=0x7f89040093e0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:45.237 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.233+0000 7f88fbfff6c0 1 -- 10.20.193.27:0/474388304 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f89100ca5d0 con 0x7f891c10f200 2026-02-13T02:22:45.351 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.349+0000 7f89214a36c0 1 -- 10.20.193.27:0/474388304 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f891c11e3b0 con 0x7f891c10f200 2026-02-13T02:22:45.351 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.349+0000 7f88fbfff6c0 1 -- 10.20.193.27:0/474388304 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f89100cd780 con 0x7f891c10f200 2026-02-13T02:22:45.352 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:22:45.352 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:22:45.353 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.350+0000 7f89214a36c0 1 -- 10.20.193.27:0/474388304 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f88f00b5710 msgr2=0x7f88f00d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:45.353 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.350+0000 7f89214a36c0 1 --2- 10.20.193.27:0/474388304 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f88f00b5710 0x7f88f00d5bc0 secure :-1 s=READY pgs=235 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f891c123bf0 tx=0x7f89040093e0 comp rx=0 tx=0).stop 2026-02-13T02:22:45.353 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.350+0000 7f89214a36c0 1 -- 10.20.193.27:0/474388304 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f891c10f200 msgr2=0x7f891c11d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:45.353 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.350+0000 7f89214a36c0 1 --2- 10.20.193.27:0/474388304 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f891c10f200 0x7f891c11d260 secure :-1 s=READY pgs=343 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f891002f230 tx=0x7f891002fc00 comp rx=0 tx=0).stop 2026-02-13T02:22:45.354 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.350+0000 7f89214a36c0 1 -- 10.20.193.27:0/474388304 shutdown_connections 2026-02-13T02:22:45.354 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.350+0000 7f89214a36c0 1 --2- 10.20.193.27:0/474388304 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f88f00b5710 0x7f88f00d5bc0 unknown :-1 s=CLOSED pgs=235 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:45.354 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.350+0000 7f89214a36c0 1 --2- 10.20.193.27:0/474388304 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f891c122b60 0x7f891c123000 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:45.354 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.350+0000 7f89214a36c0 1 --2- 10.20.193.27:0/474388304 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f891c115410 0x7f891c11d7a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:45.354 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.350+0000 7f89214a36c0 1 --2- 10.20.193.27:0/474388304 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f891c10f200 0x7f891c11d260 unknown :-1 s=CLOSED pgs=343 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:45.354 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.351+0000 7f89214a36c0 1 -- 10.20.193.27:0/474388304 >> 10.20.193.27:0/474388304 conn(0x7f891c0ecb00 msgr2=0x7f891c10cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:45.354 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.351+0000 7f89214a36c0 1 -- 10.20.193.27:0/474388304 shutdown_connections 2026-02-13T02:22:45.354 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:45.351+0000 7f89214a36c0 1 -- 10.20.193.27:0/474388304 wait complete. 2026-02-13T02:22:46.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:46 trial027 ceph-mon[24567]: pgmap v198: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 2.2 KiB/s rd, 255 B/s wr, 1 op/s 2026-02-13T02:22:46.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:46 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/474388304' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:22:46.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:46 trial028 ceph-mon[26701]: pgmap v198: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 2.2 KiB/s rd, 255 B/s wr, 1 op/s 2026-02-13T02:22:46.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:46 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/474388304' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:22:46.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:46 trial107 ceph-mon[27422]: pgmap v198: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 2.2 KiB/s rd, 255 B/s wr, 1 op/s 2026-02-13T02:22:46.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:46 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/474388304' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:22:48.395 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:22:48.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:48 trial027 ceph-mon[24567]: pgmap v199: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 2 op/s 2026-02-13T02:22:48.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:48 trial028 ceph-mon[26701]: pgmap v199: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 2 op/s 2026-02-13T02:22:48.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:48 trial107 ceph-mon[27422]: pgmap v199: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 2 op/s 2026-02-13T02:22:48.514 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:48.629 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3d1cf9d6c0 1 -- 10.20.193.27:0/3790721778 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3d18115440 msgr2=0x7f3d18175de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:48.629 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3d1cf9d6c0 1 --2- 10.20.193.27:0/3790721778 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3d18115440 0x7f3d18175de0 secure :-1 s=READY pgs=344 gs=1 cs=0 l=1 c_cookie=650285114727866223 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3d0800aa70 tx=0x7f3d0802fde0 comp rx=0 tx=0).stop 2026-02-13T02:22:48.629 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3d1cf9d6c0 1 -- 10.20.193.27:0/3790721778 shutdown_connections 2026-02-13T02:22:48.629 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3d1cf9d6c0 1 --2- 10.20.193.27:0/3790721778 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3d18115440 0x7f3d18175de0 unknown :-1 s=CLOSED pgs=344 gs=1 cs=0 l=1 c_cookie=650285114727866223 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:48.630 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3d1cf9d6c0 1 --2- 10.20.193.27:0/3790721778 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3d1810fc20 0x7f3d18114c60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:48.630 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3d1cf9d6c0 1 --2- 10.20.193.27:0/3790721778 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3d1810f230 0x7f3d1810f600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:48.630 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3d1cf9d6c0 1 -- 10.20.193.27:0/3790721778 >> 10.20.193.27:0/3790721778 conn(0x7f3d180ecb30 msgr2=0x7f3d1810cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:48.630 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3d1cf9d6c0 1 -- 10.20.193.27:0/3790721778 shutdown_connections 2026-02-13T02:22:48.630 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3d1cf9d6c0 1 -- 10.20.193.27:0/3790721778 wait complete. 2026-02-13T02:22:48.630 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3d1cf9d6c0 1 Processor -- start 2026-02-13T02:22:48.630 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3d1cf9d6c0 1 -- start start 2026-02-13T02:22:48.630 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3d1cf9d6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3d1810f230 0x7f3d1811d210 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:48.630 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3d1cf9d6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3d18115440 0x7f3d1811d750 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:48.630 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3d1cf9d6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3d18122b10 0x7f3d18122fb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:48.630 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3d1cf9d6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3d18123520 con 0x7f3d18122b10 2026-02-13T02:22:48.630 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3d1cf9d6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3d18123690 con 0x7f3d18115440 2026-02-13T02:22:48.631 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3d1cf9d6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3d18123850 con 0x7f3d1810f230 2026-02-13T02:22:48.631 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3d16d766c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3d18122b10 0x7f3d18122fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:48.631 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3d16d766c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3d18122b10 0x7f3d18122fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55910/0 (socket says 10.20.193.27:55910) 2026-02-13T02:22:48.631 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3d16d766c0 1 -- 10.20.193.27:0/323418024 learned_addr learned my addr 10.20.193.27:0/323418024 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:48.631 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3d15d746c0 1 --2- 10.20.193.27:0/323418024 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3d18115440 0x7f3d1811d750 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:48.631 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3d165756c0 1 --2- 10.20.193.27:0/323418024 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3d1810f230 0x7f3d1811d210 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:48.631 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3d16d766c0 1 -- 10.20.193.27:0/323418024 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3d1810f230 msgr2=0x7f3d1811d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:48.631 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3d16d766c0 1 --2- 10.20.193.27:0/323418024 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3d1810f230 0x7f3d1811d210 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:48.631 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3d16d766c0 1 -- 10.20.193.27:0/323418024 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3d18115440 msgr2=0x7f3d1811d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:48.631 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3d16d766c0 1 --2- 10.20.193.27:0/323418024 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3d18115440 0x7f3d1811d750 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:48.631 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3d16d766c0 1 -- 10.20.193.27:0/323418024 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0800a660 con 0x7f3d18122b10 2026-02-13T02:22:48.632 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3d165756c0 1 --2- 10.20.193.27:0/323418024 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3d1810f230 0x7f3d1811d210 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:22:48.632 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3d16d766c0 1 --2- 10.20.193.27:0/323418024 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3d18122b10 0x7f3d18122fb0 secure :-1 s=READY pgs=345 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3d080054b0 tx=0x7f3d080054e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:48.632 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3cff7fe6c0 1 -- 10.20.193.27:0/323418024 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3d08003a30 con 0x7f3d18122b10 2026-02-13T02:22:48.632 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3cff7fe6c0 1 -- 10.20.193.27:0/323418024 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3d08003b90 con 0x7f3d18122b10 2026-02-13T02:22:48.632 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3d1cf9d6c0 1 -- 10.20.193.27:0/323418024 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d18163ca0 con 0x7f3d18122b10 2026-02-13T02:22:48.632 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.625+0000 7f3cff7fe6c0 1 -- 10.20.193.27:0/323418024 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3d080382f0 con 0x7f3d18122b10 2026-02-13T02:22:48.632 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.626+0000 7f3d1cf9d6c0 1 -- 10.20.193.27:0/323418024 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d181640d0 con 0x7f3d18122b10 2026-02-13T02:22:48.632 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.626+0000 7f3d1cf9d6c0 1 -- 10.20.193.27:0/323418024 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d18110500 con 0x7f3d18122b10 2026-02-13T02:22:48.633 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.627+0000 7f3cff7fe6c0 1 -- 10.20.193.27:0/323418024 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f3d08008260 con 0x7f3d18122b10 2026-02-13T02:22:48.633 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.628+0000 7f3cff7fe6c0 1 --2- 10.20.193.27:0/323418024 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3cf00b5710 0x7f3cf00d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:48.633 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.628+0000 7f3d165756c0 1 --2- 10.20.193.27:0/323418024 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3cf00b5710 0x7f3cf00d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:48.633 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.628+0000 7f3cff7fe6c0 1 -- 10.20.193.27:0/323418024 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7f3d08108ed0 con 0x7f3d18122b10 2026-02-13T02:22:48.634 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.628+0000 7f3d165756c0 1 --2- 10.20.193.27:0/323418024 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3cf00b5710 0x7f3cf00d5bc0 secure :-1 s=READY pgs=236 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3d0c005dc0 tx=0x7f3d0c005f90 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:48.634 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.628+0000 7f3cff7fe6c0 1 -- 10.20.193.27:0/323418024 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f3d080cb8f0 con 0x7f3d18122b10 2026-02-13T02:22:48.757 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.754+0000 7f3d1cf9d6c0 1 -- 10.20.193.27:0/323418024 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3d1810f600 con 0x7f3d18122b10 2026-02-13T02:22:48.757 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.754+0000 7f3cff7fe6c0 1 -- 10.20.193.27:0/323418024 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f3d08003d00 con 0x7f3d18122b10 2026-02-13T02:22:48.757 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:22:48.757 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:22:48.758 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.755+0000 7f3d1cf9d6c0 1 -- 10.20.193.27:0/323418024 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3cf00b5710 msgr2=0x7f3cf00d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:48.758 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.756+0000 7f3d1cf9d6c0 1 --2- 10.20.193.27:0/323418024 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3cf00b5710 0x7f3cf00d5bc0 secure :-1 s=READY pgs=236 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3d0c005dc0 tx=0x7f3d0c005f90 comp rx=0 tx=0).stop 2026-02-13T02:22:48.758 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.756+0000 7f3d1cf9d6c0 1 -- 10.20.193.27:0/323418024 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3d18122b10 msgr2=0x7f3d18122fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:48.759 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.756+0000 7f3d1cf9d6c0 1 --2- 10.20.193.27:0/323418024 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3d18122b10 0x7f3d18122fb0 secure :-1 s=READY pgs=345 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3d080054b0 tx=0x7f3d080054e0 comp rx=0 tx=0).stop 2026-02-13T02:22:48.759 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.756+0000 7f3d1cf9d6c0 1 -- 10.20.193.27:0/323418024 shutdown_connections 2026-02-13T02:22:48.759 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.756+0000 7f3d1cf9d6c0 1 --2- 10.20.193.27:0/323418024 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3cf00b5710 0x7f3cf00d5bc0 unknown :-1 s=CLOSED pgs=236 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:48.759 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.756+0000 7f3d1cf9d6c0 1 --2- 10.20.193.27:0/323418024 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3d18122b10 0x7f3d18122fb0 unknown :-1 s=CLOSED pgs=345 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:48.759 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.756+0000 7f3d1cf9d6c0 1 --2- 10.20.193.27:0/323418024 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3d18115440 0x7f3d1811d750 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:48.759 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.756+0000 7f3d1cf9d6c0 1 --2- 10.20.193.27:0/323418024 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3d1810f230 0x7f3d1811d210 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:48.759 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.756+0000 7f3d1cf9d6c0 1 -- 10.20.193.27:0/323418024 >> 10.20.193.27:0/323418024 conn(0x7f3d180ecb30 msgr2=0x7f3d1810cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:48.759 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.756+0000 7f3d1cf9d6c0 1 -- 10.20.193.27:0/323418024 shutdown_connections 2026-02-13T02:22:48.759 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:48.756+0000 7f3d1cf9d6c0 1 -- 10.20.193.27:0/323418024 wait complete. 2026-02-13T02:22:49.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:49 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/323418024' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:22:49.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:49 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/323418024' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:22:49.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:49 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/323418024' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:22:50.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:50 trial027 ceph-mon[24567]: pgmap v200: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2026-02-13T02:22:50.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:50 trial028 ceph-mon[26701]: pgmap v200: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2026-02-13T02:22:50.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:50 trial107 ceph-mon[27422]: pgmap v200: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2026-02-13T02:22:51.803 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:22:51.922 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:52.039 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.035+0000 7fa4fa1486c0 1 -- 10.20.193.27:0/4083288582 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa4f410b120 msgr2=0x7fa4f410b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:52.039 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.035+0000 7fa4fa1486c0 1 --2- 10.20.193.27:0/4083288582 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa4f410b120 0x7fa4f410b550 secure :-1 s=READY pgs=346 gs=1 cs=0 l=1 c_cookie=8717884053602329419 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa4e4009970 tx=0x7fa4e402ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:52.040 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.035+0000 7fa4fa1486c0 1 -- 10.20.193.27:0/4083288582 shutdown_connections 2026-02-13T02:22:52.040 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.035+0000 7fa4fa1486c0 1 --2- 10.20.193.27:0/4083288582 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa4f410bd00 0x7fa4f4175a70 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:52.040 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.035+0000 7fa4fa1486c0 1 --2- 10.20.193.27:0/4083288582 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa4f410b120 0x7fa4f410b550 unknown :-1 s=CLOSED pgs=346 gs=1 cs=0 l=1 c_cookie=8717884053602329419 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:52.040 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.035+0000 7fa4fa1486c0 1 --2- 10.20.193.27:0/4083288582 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa4f410dad0 0x7fa4f410dea0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:52.040 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.035+0000 7fa4fa1486c0 1 -- 10.20.193.27:0/4083288582 >> 10.20.193.27:0/4083288582 conn(0x7fa4f4062e00 msgr2=0x7fa4f4109660 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:52.040 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.035+0000 7fa4fa1486c0 1 -- 10.20.193.27:0/4083288582 shutdown_connections 2026-02-13T02:22:52.040 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.035+0000 7fa4fa1486c0 1 -- 10.20.193.27:0/4083288582 wait complete. 2026-02-13T02:22:52.040 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.035+0000 7fa4fa1486c0 1 Processor -- start 2026-02-13T02:22:52.040 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.035+0000 7fa4fa1486c0 1 -- start start 2026-02-13T02:22:52.040 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.035+0000 7fa4fa1486c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa4f410b120 0x7fa4f40f5330 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:52.040 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.035+0000 7fa4fa1486c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa4f410bd00 0x7fa4f40f5870 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:52.040 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.035+0000 7fa4fa1486c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa4f410dad0 0x7fa4f40fac30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:52.041 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.035+0000 7fa4fa1486c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f40fb280 con 0x7fa4f410bd00 2026-02-13T02:22:52.041 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.035+0000 7fa4fa1486c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f40fb3f0 con 0x7fa4f410b120 2026-02-13T02:22:52.041 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.035+0000 7fa4fa1486c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f40fb5b0 con 0x7fa4f410dad0 2026-02-13T02:22:52.041 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.035+0000 7fa4f2ffd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa4f410bd00 0x7fa4f40f5870 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:52.041 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.035+0000 7fa4f2ffd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa4f410bd00 0x7fa4f40f5870 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55928/0 (socket says 10.20.193.27:55928) 2026-02-13T02:22:52.041 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.035+0000 7fa4f2ffd6c0 1 -- 10.20.193.27:0/2804080670 learned_addr learned my addr 10.20.193.27:0/2804080670 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:52.041 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.035+0000 7fa4f37fe6c0 1 --2- 10.20.193.27:0/2804080670 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa4f410b120 0x7fa4f40f5330 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:52.041 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.036+0000 7fa4f3fff6c0 1 --2- 10.20.193.27:0/2804080670 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa4f410dad0 0x7fa4f40fac30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:52.041 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.036+0000 7fa4f2ffd6c0 1 -- 10.20.193.27:0/2804080670 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa4f410dad0 msgr2=0x7fa4f40fac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:52.041 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.036+0000 7fa4f2ffd6c0 1 --2- 10.20.193.27:0/2804080670 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa4f410dad0 0x7fa4f40fac30 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:52.041 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.036+0000 7fa4f2ffd6c0 1 -- 10.20.193.27:0/2804080670 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa4f410b120 msgr2=0x7fa4f40f5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:52.041 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.036+0000 7fa4f2ffd6c0 1 --2- 10.20.193.27:0/2804080670 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa4f410b120 0x7fa4f40f5330 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:52.042 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.036+0000 7fa4f2ffd6c0 1 -- 10.20.193.27:0/2804080670 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e4009590 con 0x7fa4f410bd00 2026-02-13T02:22:52.042 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.036+0000 7fa4f37fe6c0 1 --2- 10.20.193.27:0/2804080670 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa4f410b120 0x7fa4f40f5330 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:22:52.042 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.036+0000 7fa4f2ffd6c0 1 --2- 10.20.193.27:0/2804080670 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa4f410bd00 0x7fa4f40f5870 secure :-1 s=READY pgs=347 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa4e40092b0 tx=0x7fa4e402fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:52.042 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.036+0000 7fa4f0ff96c0 1 -- 10.20.193.27:0/2804080670 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa4e4002ba0 con 0x7fa4f410bd00 2026-02-13T02:22:52.042 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.036+0000 7fa4fa1486c0 1 -- 10.20.193.27:0/2804080670 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f40fb830 con 0x7fa4f410bd00 2026-02-13T02:22:52.042 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.036+0000 7fa4f0ff96c0 1 -- 10.20.193.27:0/2804080670 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa4e4007680 con 0x7fa4f410bd00 2026-02-13T02:22:52.042 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.036+0000 7fa4f0ff96c0 1 -- 10.20.193.27:0/2804080670 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa4e4037520 con 0x7fa4f410bd00 2026-02-13T02:22:52.042 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.036+0000 7fa4fa1486c0 1 -- 10.20.193.27:0/2804080670 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f4172bf0 con 0x7fa4f410bd00 2026-02-13T02:22:52.042 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.036+0000 7fa4fa1486c0 1 -- 10.20.193.27:0/2804080670 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4f410c130 con 0x7fa4f410bd00 2026-02-13T02:22:52.043 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.038+0000 7fa4f0ff96c0 1 -- 10.20.193.27:0/2804080670 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7fa4e4002d00 con 0x7fa4f410bd00 2026-02-13T02:22:52.043 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.038+0000 7fa4f0ff96c0 1 --2- 10.20.193.27:0/2804080670 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa4c80b5690 0x7fa4c80d5b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:52.043 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.038+0000 7fa4f37fe6c0 1 --2- 10.20.193.27:0/2804080670 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa4c80b5690 0x7fa4c80d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:52.044 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.038+0000 7fa4f0ff96c0 1 -- 10.20.193.27:0/2804080670 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7fa4e4107c70 con 0x7fa4f410bd00 2026-02-13T02:22:52.044 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.039+0000 7fa4f0ff96c0 1 -- 10.20.193.27:0/2804080670 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fa4e40cd660 con 0x7fa4f410bd00 2026-02-13T02:22:52.044 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.039+0000 7fa4f37fe6c0 1 --2- 10.20.193.27:0/2804080670 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa4c80b5690 0x7fa4c80d5b40 secure :-1 s=READY pgs=237 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa4e0005fd0 tx=0x7fa4e0009650 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:52.155 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.152+0000 7fa4fa1486c0 1 -- 10.20.193.27:0/2804080670 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa4f40f6480 con 0x7fa4f410bd00 2026-02-13T02:22:52.155 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:22:52.156 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:22:52.157 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.153+0000 7fa4f0ff96c0 1 -- 10.20.193.27:0/2804080670 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7fa4e40cd840 con 0x7fa4f410bd00 2026-02-13T02:22:52.157 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.154+0000 7fa4fa1486c0 1 -- 10.20.193.27:0/2804080670 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa4c80b5690 msgr2=0x7fa4c80d5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:52.157 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.154+0000 7fa4fa1486c0 1 --2- 10.20.193.27:0/2804080670 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa4c80b5690 0x7fa4c80d5b40 secure :-1 s=READY pgs=237 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa4e0005fd0 tx=0x7fa4e0009650 comp rx=0 tx=0).stop 2026-02-13T02:22:52.157 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.154+0000 7fa4fa1486c0 1 -- 10.20.193.27:0/2804080670 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa4f410bd00 msgr2=0x7fa4f40f5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:52.157 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.154+0000 7fa4fa1486c0 1 --2- 10.20.193.27:0/2804080670 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa4f410bd00 0x7fa4f40f5870 secure :-1 s=READY pgs=347 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa4e40092b0 tx=0x7fa4e402fd40 comp rx=0 tx=0).stop 2026-02-13T02:22:52.157 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.154+0000 7fa4fa1486c0 1 -- 10.20.193.27:0/2804080670 shutdown_connections 2026-02-13T02:22:52.157 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.154+0000 7fa4fa1486c0 1 --2- 10.20.193.27:0/2804080670 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa4c80b5690 0x7fa4c80d5b40 unknown :-1 s=CLOSED pgs=237 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:52.157 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.154+0000 7fa4fa1486c0 1 --2- 10.20.193.27:0/2804080670 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa4f410dad0 0x7fa4f40fac30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:52.157 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.154+0000 7fa4fa1486c0 1 --2- 10.20.193.27:0/2804080670 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa4f410bd00 0x7fa4f40f5870 unknown :-1 s=CLOSED pgs=347 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:52.158 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.154+0000 7fa4fa1486c0 1 --2- 10.20.193.27:0/2804080670 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa4f410b120 0x7fa4f40f5330 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:52.158 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.154+0000 7fa4fa1486c0 1 -- 10.20.193.27:0/2804080670 >> 10.20.193.27:0/2804080670 conn(0x7fa4f4062e00 msgr2=0x7fa4f40eba50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:52.158 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.154+0000 7fa4fa1486c0 1 -- 10.20.193.27:0/2804080670 shutdown_connections 2026-02-13T02:22:52.158 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:52.154+0000 7fa4fa1486c0 1 -- 10.20.193.27:0/2804080670 wait complete. 2026-02-13T02:22:52.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:52 trial027 ceph-mon[24567]: pgmap v201: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2026-02-13T02:22:52.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:52 trial028 ceph-mon[26701]: pgmap v201: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2026-02-13T02:22:52.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:52 trial107 ceph-mon[27422]: pgmap v201: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2026-02-13T02:22:53.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:53 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2804080670' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:22:53.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:53 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2804080670' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:22:53.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:53 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2804080670' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:22:54.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:54 trial027 ceph-mon[24567]: pgmap v202: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2026-02-13T02:22:54.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:54 trial028 ceph-mon[26701]: pgmap v202: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2026-02-13T02:22:54.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:54 trial107 ceph-mon[27422]: pgmap v202: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2026-02-13T02:22:55.198 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:22:55.316 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:55.432 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.431+0000 7febc4a046c0 1 -- 10.20.193.27:0/2659894928 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7febc010cfb0 msgr2=0x7febc010d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:55.432 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.431+0000 7febc4a046c0 1 --2- 10.20.193.27:0/2659894928 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7febc010cfb0 0x7febc010d380 secure :-1 s=READY pgs=348 gs=1 cs=0 l=1 c_cookie=4376838895068733018 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7febb40099a0 tx=0x7febb402ed20 comp rx=0 tx=0).stop 2026-02-13T02:22:55.432 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.431+0000 7febc4a046c0 1 -- 10.20.193.27:0/2659894928 shutdown_connections 2026-02-13T02:22:55.432 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.431+0000 7febc4a046c0 1 --2- 10.20.193.27:0/2659894928 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7febc01131c0 0x7febc0173b60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:55.432 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.431+0000 7febc4a046c0 1 --2- 10.20.193.27:0/2659894928 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7febc010d9a0 0x7febc01129e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:55.432 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.431+0000 7febc4a046c0 1 --2- 10.20.193.27:0/2659894928 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7febc010cfb0 0x7febc010d380 unknown :-1 s=CLOSED pgs=348 gs=1 cs=0 l=1 c_cookie=4376838895068733018 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:55.432 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.431+0000 7febc4a046c0 1 -- 10.20.193.27:0/2659894928 >> 10.20.193.27:0/2659894928 conn(0x7febc00ea8b0 msgr2=0x7febc010acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:55.432 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.431+0000 7febc4a046c0 1 -- 10.20.193.27:0/2659894928 shutdown_connections 2026-02-13T02:22:55.432 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.431+0000 7febc4a046c0 1 -- 10.20.193.27:0/2659894928 wait complete. 2026-02-13T02:22:55.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.431+0000 7febc4a046c0 1 Processor -- start 2026-02-13T02:22:55.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.431+0000 7febc4a046c0 1 -- start start 2026-02-13T02:22:55.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.431+0000 7febc4a046c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7febc010cfb0 0x7febc0060700 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:55.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.431+0000 7febc4a046c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7febc01131c0 0x7febc0060c40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:55.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.431+0000 7febc4a046c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7febc0061180 0x7febc01633a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:55.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.431+0000 7febc4a046c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7febc01638e0 con 0x7febc0061180 2026-02-13T02:22:55.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.431+0000 7febc4a046c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7febc0163a50 con 0x7febc01131c0 2026-02-13T02:22:55.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.431+0000 7febc4a046c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7febc0163c10 con 0x7febc010cfb0 2026-02-13T02:22:55.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.431+0000 7febbed766c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7febc0061180 0x7febc01633a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:55.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.431+0000 7febbed766c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7febc0061180 0x7febc01633a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55944/0 (socket says 10.20.193.27:55944) 2026-02-13T02:22:55.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.431+0000 7febbed766c0 1 -- 10.20.193.27:0/2263347746 learned_addr learned my addr 10.20.193.27:0/2263347746 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:55.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.431+0000 7febbdd746c0 1 --2- 10.20.193.27:0/2263347746 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7febc01131c0 0x7febc0060c40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:55.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.431+0000 7febbe5756c0 1 --2- 10.20.193.27:0/2263347746 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7febc010cfb0 0x7febc0060700 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:55.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.431+0000 7febbed766c0 1 -- 10.20.193.27:0/2263347746 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7febc010cfb0 msgr2=0x7febc0060700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:55.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.431+0000 7febbed766c0 1 --2- 10.20.193.27:0/2263347746 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7febc010cfb0 0x7febc0060700 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:55.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.431+0000 7febbed766c0 1 -- 10.20.193.27:0/2263347746 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7febc01131c0 msgr2=0x7febc0060c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:55.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.431+0000 7febbed766c0 1 --2- 10.20.193.27:0/2263347746 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7febc01131c0 0x7febc0060c40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:55.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.431+0000 7febbed766c0 1 -- 10.20.193.27:0/2263347746 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb4009590 con 0x7febc0061180 2026-02-13T02:22:55.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.431+0000 7febbed766c0 1 --2- 10.20.193.27:0/2263347746 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7febc0061180 0x7febc01633a0 secure :-1 s=READY pgs=349 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7febb0006ea0 tx=0x7febb000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:55.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.432+0000 7feba77fe6c0 1 -- 10.20.193.27:0/2263347746 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7febb0003a30 con 0x7febc0061180 2026-02-13T02:22:55.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.432+0000 7febc4a046c0 1 -- 10.20.193.27:0/2263347746 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc0163ef0 con 0x7febc0061180 2026-02-13T02:22:55.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.432+0000 7feba77fe6c0 1 -- 10.20.193.27:0/2263347746 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7febb00054f0 con 0x7febc0061180 2026-02-13T02:22:55.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.432+0000 7feba77fe6c0 1 -- 10.20.193.27:0/2263347746 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7febb000dcd0 con 0x7febc0061180 2026-02-13T02:22:55.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.432+0000 7febc4a046c0 1 -- 10.20.193.27:0/2263347746 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc01643d0 con 0x7febc0061180 2026-02-13T02:22:55.434 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.432+0000 7febc4a046c0 1 -- 10.20.193.27:0/2263347746 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febc010e280 con 0x7febc0061180 2026-02-13T02:22:55.435 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.434+0000 7feba77fe6c0 1 -- 10.20.193.27:0/2263347746 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7febb0005030 con 0x7febc0061180 2026-02-13T02:22:55.435 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.434+0000 7feba77fe6c0 1 --2- 10.20.193.27:0/2263347746 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7feb980b5690 0x7feb980d5b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:55.435 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.434+0000 7feba77fe6c0 1 -- 10.20.193.27:0/2263347746 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7762+0+0 (secure 0 0 0) 0x7febb00e4b70 con 0x7febc0061180 2026-02-13T02:22:55.435 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.434+0000 7febbe5756c0 1 --2- 10.20.193.27:0/2263347746 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7feb980b5690 0x7feb980d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:55.435 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.435+0000 7feba77fe6c0 1 -- 10.20.193.27:0/2263347746 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7febb011d5e0 con 0x7febc0061180 2026-02-13T02:22:55.436 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.435+0000 7febbe5756c0 1 --2- 10.20.193.27:0/2263347746 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7feb980b5690 0x7feb980d5b40 secure :-1 s=READY pgs=238 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7febb40092e0 tx=0x7febb4005c00 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:55.549 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.548+0000 7febc4a046c0 1 -- 10.20.193.27:0/2263347746 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7febc010d380 con 0x7febc0061180 2026-02-13T02:22:55.549 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.548+0000 7feba77fe6c0 1 -- 10.20.193.27:0/2263347746 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7febb00a7590 con 0x7febc0061180 2026-02-13T02:22:55.549 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:22:55.549 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:22:55.550 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.549+0000 7febc4a046c0 1 -- 10.20.193.27:0/2263347746 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7feb980b5690 msgr2=0x7feb980d5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:55.550 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.549+0000 7febc4a046c0 1 --2- 10.20.193.27:0/2263347746 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7feb980b5690 0x7feb980d5b40 secure :-1 s=READY pgs=238 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7febb40092e0 tx=0x7febb4005c00 comp rx=0 tx=0).stop 2026-02-13T02:22:55.550 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.549+0000 7febc4a046c0 1 -- 10.20.193.27:0/2263347746 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7febc0061180 msgr2=0x7febc01633a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:55.550 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.549+0000 7febc4a046c0 1 --2- 10.20.193.27:0/2263347746 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7febc0061180 0x7febc01633a0 secure :-1 s=READY pgs=349 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7febb0006ea0 tx=0x7febb000cca0 comp rx=0 tx=0).stop 2026-02-13T02:22:55.550 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.549+0000 7febc4a046c0 1 -- 10.20.193.27:0/2263347746 shutdown_connections 2026-02-13T02:22:55.550 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.549+0000 7febc4a046c0 1 --2- 10.20.193.27:0/2263347746 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7feb980b5690 0x7feb980d5b40 unknown :-1 s=CLOSED pgs=238 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:55.550 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.549+0000 7febc4a046c0 1 --2- 10.20.193.27:0/2263347746 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7febc0061180 0x7febc01633a0 unknown :-1 s=CLOSED pgs=349 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:55.550 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.549+0000 7febc4a046c0 1 --2- 10.20.193.27:0/2263347746 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7febc01131c0 0x7febc0060c40 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:55.550 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.549+0000 7febc4a046c0 1 --2- 10.20.193.27:0/2263347746 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7febc010cfb0 0x7febc0060700 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:55.550 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.549+0000 7febc4a046c0 1 -- 10.20.193.27:0/2263347746 >> 10.20.193.27:0/2263347746 conn(0x7febc00ea8b0 msgr2=0x7febc010ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:55.550 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.549+0000 7febc4a046c0 1 -- 10.20.193.27:0/2263347746 shutdown_connections 2026-02-13T02:22:55.551 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:55.549+0000 7febc4a046c0 1 -- 10.20.193.27:0/2263347746 wait complete. 2026-02-13T02:22:56.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:56 trial027 ceph-mon[24567]: pgmap v203: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1023 B/s rd, 1 op/s 2026-02-13T02:22:56.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:56 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2263347746' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:22:56.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:56 trial028 ceph-mon[26701]: pgmap v203: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1023 B/s rd, 1 op/s 2026-02-13T02:22:56.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:56 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2263347746' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:22:56.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:56 trial107 ceph-mon[27422]: pgmap v203: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1023 B/s rd, 1 op/s 2026-02-13T02:22:56.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:56 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2263347746' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:22:57.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]} : dispatch 2026-02-13T02:22:57.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]} : dispatch 2026-02-13T02:22:57.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 7]} : dispatch 2026-02-13T02:22:57.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]} : dispatch 2026-02-13T02:22:57.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]} : dispatch 2026-02-13T02:22:57.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]} : dispatch 2026-02-13T02:22:57.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]} : dispatch 2026-02-13T02:22:57.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6, 3, 8]} : dispatch 2026-02-13T02:22:57.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:57 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]} : dispatch 2026-02-13T02:22:57.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:57 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]} : dispatch 2026-02-13T02:22:57.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:57 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]} : dispatch 2026-02-13T02:22:57.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:57 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 7]} : dispatch 2026-02-13T02:22:57.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:57 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]} : dispatch 2026-02-13T02:22:57.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:57 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]} : dispatch 2026-02-13T02:22:57.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:57 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]} : dispatch 2026-02-13T02:22:57.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:57 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]} : dispatch 2026-02-13T02:22:57.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:57 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6, 3, 8]} : dispatch 2026-02-13T02:22:57.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:57 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]} : dispatch 2026-02-13T02:22:57.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:57 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]} : dispatch 2026-02-13T02:22:57.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:57 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]} : dispatch 2026-02-13T02:22:57.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:57 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 7]} : dispatch 2026-02-13T02:22:57.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:57 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]} : dispatch 2026-02-13T02:22:57.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:57 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]} : dispatch 2026-02-13T02:22:57.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:57 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]} : dispatch 2026-02-13T02:22:57.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:57 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]} : dispatch 2026-02-13T02:22:57.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:57 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6, 3, 8]} : dispatch 2026-02-13T02:22:57.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:57 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]} : dispatch 2026-02-13T02:22:58.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:58 trial027 ceph-mon[24567]: pgmap v204: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.4 KiB/s rd, 2 op/s 2026-02-13T02:22:58.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2026-02-13T02:22:58.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2026-02-13T02:22:58.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 7]}]': finished 2026-02-13T02:22:58.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2026-02-13T02:22:58.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2026-02-13T02:22:58.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2026-02-13T02:22:58.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2026-02-13T02:22:58.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6, 3, 8]}]': finished 2026-02-13T02:22:58.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:58 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2026-02-13T02:22:58.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:58 trial027 ceph-mon[24567]: osdmap e71: 12 total, 12 up, 12 in 2026-02-13T02:22:58.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:58 trial028 ceph-mon[26701]: pgmap v204: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.4 KiB/s rd, 2 op/s 2026-02-13T02:22:58.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2026-02-13T02:22:58.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2026-02-13T02:22:58.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 7]}]': finished 2026-02-13T02:22:58.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2026-02-13T02:22:58.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2026-02-13T02:22:58.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2026-02-13T02:22:58.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2026-02-13T02:22:58.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6, 3, 8]}]': finished 2026-02-13T02:22:58.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:58 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2026-02-13T02:22:58.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:58 trial028 ceph-mon[26701]: osdmap e71: 12 total, 12 up, 12 in 2026-02-13T02:22:58.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:58 trial107 ceph-mon[27422]: pgmap v204: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.4 KiB/s rd, 2 op/s 2026-02-13T02:22:58.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2026-02-13T02:22:58.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2026-02-13T02:22:58.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 7]}]': finished 2026-02-13T02:22:58.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2026-02-13T02:22:58.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2026-02-13T02:22:58.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2026-02-13T02:22:58.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2026-02-13T02:22:58.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6, 3, 8]}]': finished 2026-02-13T02:22:58.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:58 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2026-02-13T02:22:58.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:58 trial107 ceph-mon[27422]: osdmap e71: 12 total, 12 up, 12 in 2026-02-13T02:22:58.592 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:22:58.710 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:22:58.822 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.821+0000 7f24a09486c0 1 -- 10.20.193.27:0/439664746 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2498115440 msgr2=0x7f2498175de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:58.822 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.821+0000 7f24a09486c0 1 --2- 10.20.193.27:0/439664746 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2498115440 0x7f2498175de0 secure :-1 s=READY pgs=350 gs=1 cs=0 l=1 c_cookie=11971091706870195239 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f248c00aa70 tx=0x7f248c02fde0 comp rx=0 tx=0).stop 2026-02-13T02:22:58.822 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.821+0000 7f24a09486c0 1 -- 10.20.193.27:0/439664746 shutdown_connections 2026-02-13T02:22:58.823 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.821+0000 7f24a09486c0 1 --2- 10.20.193.27:0/439664746 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2498115440 0x7f2498175de0 unknown :-1 s=CLOSED pgs=350 gs=1 cs=0 l=1 c_cookie=11971091706870195239 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:58.823 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.821+0000 7f24a09486c0 1 --2- 10.20.193.27:0/439664746 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f249810fc20 0x7f2498114c60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:58.823 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.821+0000 7f24a09486c0 1 --2- 10.20.193.27:0/439664746 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f249810f230 0x7f249810f600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:58.823 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.821+0000 7f24a09486c0 1 -- 10.20.193.27:0/439664746 >> 10.20.193.27:0/439664746 conn(0x7f24980ecb30 msgr2=0x7f249810cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:58.823 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.821+0000 7f24a09486c0 1 -- 10.20.193.27:0/439664746 shutdown_connections 2026-02-13T02:22:58.823 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.821+0000 7f24a09486c0 1 -- 10.20.193.27:0/439664746 wait complete. 2026-02-13T02:22:58.823 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.821+0000 7f24a09486c0 1 Processor -- start 2026-02-13T02:22:58.823 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.821+0000 7f24a09486c0 1 -- start start 2026-02-13T02:22:58.823 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.822+0000 7f24a09486c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f249810f230 0x7f249811d2a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:58.823 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.822+0000 7f24a09486c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2498115440 0x7f249811d7e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:58.823 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.822+0000 7f24a09486c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2498122ba0 0x7f2498123040 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:58.823 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.822+0000 7f24a09486c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f24981235b0 con 0x7f2498115440 2026-02-13T02:22:58.823 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.822+0000 7f24a09486c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2498123720 con 0x7f249810f230 2026-02-13T02:22:58.823 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.822+0000 7f24a09486c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f24981238e0 con 0x7f2498122ba0 2026-02-13T02:22:58.824 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.822+0000 7f249debc6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2498115440 0x7f249811d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:58.824 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.822+0000 7f249debc6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2498115440 0x7f249811d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:57828/0 (socket says 10.20.193.27:57828) 2026-02-13T02:22:58.824 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.822+0000 7f249debc6c0 1 -- 10.20.193.27:0/555510106 learned_addr learned my addr 10.20.193.27:0/555510106 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:22:58.824 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.822+0000 7f249e6bd6c0 1 --2- 10.20.193.27:0/555510106 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f249810f230 0x7f249811d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:58.824 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.822+0000 7f249eebe6c0 1 --2- 10.20.193.27:0/555510106 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2498122ba0 0x7f2498123040 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:58.824 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.822+0000 7f249debc6c0 1 -- 10.20.193.27:0/555510106 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2498122ba0 msgr2=0x7f2498123040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:58.824 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.822+0000 7f249debc6c0 1 --2- 10.20.193.27:0/555510106 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2498122ba0 0x7f2498123040 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:58.824 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.822+0000 7f249debc6c0 1 -- 10.20.193.27:0/555510106 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f249810f230 msgr2=0x7f249811d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:58.824 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.822+0000 7f249debc6c0 1 --2- 10.20.193.27:0/555510106 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f249810f230 0x7f249811d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:58.824 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.822+0000 7f249debc6c0 1 -- 10.20.193.27:0/555510106 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248c00a660 con 0x7f2498115440 2026-02-13T02:22:58.824 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.822+0000 7f249debc6c0 1 --2- 10.20.193.27:0/555510106 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2498115440 0x7f249811d7e0 secure :-1 s=READY pgs=351 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2484005dc0 tx=0x7f248400bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:58.824 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.822+0000 7f24837fe6c0 1 -- 10.20.193.27:0/555510106 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2484004250 con 0x7f2498115440 2026-02-13T02:22:58.824 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.822+0000 7f24837fe6c0 1 -- 10.20.193.27:0/555510106 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f2484009070 con 0x7f2498115440 2026-02-13T02:22:58.824 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.822+0000 7f24a09486c0 1 -- 10.20.193.27:0/555510106 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2498163d90 con 0x7f2498115440 2026-02-13T02:22:58.824 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.822+0000 7f24837fe6c0 1 -- 10.20.193.27:0/555510106 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f248400cd50 con 0x7f2498115440 2026-02-13T02:22:58.824 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.822+0000 7f24a09486c0 1 -- 10.20.193.27:0/555510106 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2498164270 con 0x7f2498115440 2026-02-13T02:22:58.824 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.822+0000 7f24a09486c0 1 -- 10.20.193.27:0/555510106 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2498110500 con 0x7f2498115440 2026-02-13T02:22:58.825 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.824+0000 7f24837fe6c0 1 -- 10.20.193.27:0/555510106 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f24840028c0 con 0x7f2498115440 2026-02-13T02:22:58.825 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.824+0000 7f24837fe6c0 1 --2- 10.20.193.27:0/555510106 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f24740b5710 0x7f24740d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:22:58.825 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.824+0000 7f249e6bd6c0 1 --2- 10.20.193.27:0/555510106 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f24740b5710 0x7f24740d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:22:58.826 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.825+0000 7f24837fe6c0 1 -- 10.20.193.27:0/555510106 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f24840e3b80 con 0x7f2498115440 2026-02-13T02:22:58.826 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.825+0000 7f249e6bd6c0 1 --2- 10.20.193.27:0/555510106 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f24740b5710 0x7f24740d5bc0 secure :-1 s=READY pgs=239 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2490005980 tx=0x7f2490005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:22:58.826 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.825+0000 7f24837fe6c0 1 -- 10.20.193.27:0/555510106 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f24840a64a0 con 0x7f2498115440 2026-02-13T02:22:58.940 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.939+0000 7f24a09486c0 1 -- 10.20.193.27:0/555510106 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f249810f600 con 0x7f2498115440 2026-02-13T02:22:58.940 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.939+0000 7f24837fe6c0 1 -- 10.20.193.27:0/555510106 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f24840a9650 con 0x7f2498115440 2026-02-13T02:22:58.940 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:22:58.940 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:22:58.941 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.940+0000 7f24a09486c0 1 -- 10.20.193.27:0/555510106 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f24740b5710 msgr2=0x7f24740d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:58.941 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.940+0000 7f24a09486c0 1 --2- 10.20.193.27:0/555510106 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f24740b5710 0x7f24740d5bc0 secure :-1 s=READY pgs=239 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2490005980 tx=0x7f2490005910 comp rx=0 tx=0).stop 2026-02-13T02:22:58.941 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.940+0000 7f24a09486c0 1 -- 10.20.193.27:0/555510106 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2498115440 msgr2=0x7f249811d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:22:58.941 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.940+0000 7f24a09486c0 1 --2- 10.20.193.27:0/555510106 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2498115440 0x7f249811d7e0 secure :-1 s=READY pgs=351 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2484005dc0 tx=0x7f248400bbb0 comp rx=0 tx=0).stop 2026-02-13T02:22:58.941 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.940+0000 7f24a09486c0 1 -- 10.20.193.27:0/555510106 shutdown_connections 2026-02-13T02:22:58.941 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.940+0000 7f24a09486c0 1 --2- 10.20.193.27:0/555510106 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f24740b5710 0x7f24740d5bc0 unknown :-1 s=CLOSED pgs=239 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:58.941 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.940+0000 7f24a09486c0 1 --2- 10.20.193.27:0/555510106 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2498122ba0 0x7f2498123040 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:58.941 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.940+0000 7f24a09486c0 1 --2- 10.20.193.27:0/555510106 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2498115440 0x7f249811d7e0 unknown :-1 s=CLOSED pgs=351 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:58.941 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.940+0000 7f24a09486c0 1 --2- 10.20.193.27:0/555510106 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f249810f230 0x7f249811d2a0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:22:58.941 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.940+0000 7f24a09486c0 1 -- 10.20.193.27:0/555510106 >> 10.20.193.27:0/555510106 conn(0x7f24980ecb30 msgr2=0x7f249810cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:22:58.941 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.940+0000 7f24a09486c0 1 -- 10.20.193.27:0/555510106 shutdown_connections 2026-02-13T02:22:58.942 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:22:58.940+0000 7f24a09486c0 1 -- 10.20.193.27:0/555510106 wait complete. 2026-02-13T02:22:59.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:59 trial027 ceph-mon[24567]: osdmap e72: 12 total, 12 up, 12 in 2026-02-13T02:22:59.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:22:59 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/555510106' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:22:59.416 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:59 trial028 ceph-mon[26701]: osdmap e72: 12 total, 12 up, 12 in 2026-02-13T02:22:59.416 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:22:59 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/555510106' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:22:59.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:59 trial107 ceph-mon[27422]: osdmap e72: 12 total, 12 up, 12 in 2026-02-13T02:22:59.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:22:59 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/555510106' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:00.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:00 trial027 ceph-mon[24567]: pgmap v207: 97 pgs: 2 peering, 95 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:23:00.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:00 trial028 ceph-mon[26701]: pgmap v207: 97 pgs: 2 peering, 95 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:23:00.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:00 trial107 ceph-mon[27422]: pgmap v207: 97 pgs: 2 peering, 95 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:23:01.981 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:23:02.099 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:23:02.217 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.214+0000 7f78cc8756c0 1 -- 10.20.193.27:0/729815472 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f78c4115440 msgr2=0x7f78c4175de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:02.217 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.214+0000 7f78cc8756c0 1 --2- 10.20.193.27:0/729815472 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f78c4115440 0x7f78c4175de0 secure :-1 s=READY pgs=352 gs=1 cs=0 l=1 c_cookie=4908306990908231040 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f78b800aa70 tx=0x7f78b802fde0 comp rx=0 tx=0).stop 2026-02-13T02:23:02.217 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.214+0000 7f78cc8756c0 1 -- 10.20.193.27:0/729815472 shutdown_connections 2026-02-13T02:23:02.217 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.214+0000 7f78cc8756c0 1 --2- 10.20.193.27:0/729815472 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f78c4115440 0x7f78c4175de0 unknown :-1 s=CLOSED pgs=352 gs=1 cs=0 l=1 c_cookie=4908306990908231040 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:02.218 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.214+0000 7f78cc8756c0 1 --2- 10.20.193.27:0/729815472 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f78c410fc20 0x7f78c4114c60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:02.218 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.214+0000 7f78cc8756c0 1 --2- 10.20.193.27:0/729815472 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f78c410f230 0x7f78c410f600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:02.218 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.214+0000 7f78cc8756c0 1 -- 10.20.193.27:0/729815472 >> 10.20.193.27:0/729815472 conn(0x7f78c40ecb30 msgr2=0x7f78c410cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:02.218 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.214+0000 7f78cc8756c0 1 -- 10.20.193.27:0/729815472 shutdown_connections 2026-02-13T02:23:02.218 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.214+0000 7f78cc8756c0 1 -- 10.20.193.27:0/729815472 wait complete. 2026-02-13T02:23:02.218 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.214+0000 7f78cc8756c0 1 Processor -- start 2026-02-13T02:23:02.218 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.214+0000 7f78cc8756c0 1 -- start start 2026-02-13T02:23:02.218 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.214+0000 7f78cc8756c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f78c410f230 0x7f78c411d1e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:02.218 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.214+0000 7f78cc8756c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f78c4115440 0x7f78c411d720 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:02.218 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.214+0000 7f78cc8756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f78c4122ae0 0x7f78c4122f80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:02.218 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.214+0000 7f78cc8756c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f78c41234f0 con 0x7f78c4122ae0 2026-02-13T02:23:02.219 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.214+0000 7f78cc8756c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f78c4123660 con 0x7f78c410f230 2026-02-13T02:23:02.219 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.214+0000 7f78cc8756c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f78c4123820 con 0x7f78c4115440 2026-02-13T02:23:02.219 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.214+0000 7f78cade76c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f78c4122ae0 0x7f78c4122f80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:02.219 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.214+0000 7f78cade76c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f78c4122ae0 0x7f78c4122f80 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:57844/0 (socket says 10.20.193.27:57844) 2026-02-13T02:23:02.219 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.214+0000 7f78cade76c0 1 -- 10.20.193.27:0/231659655 learned_addr learned my addr 10.20.193.27:0/231659655 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:23:02.219 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.214+0000 7f78ca5e66c0 1 --2- 10.20.193.27:0/231659655 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f78c410f230 0x7f78c411d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:02.219 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.214+0000 7f78c9de56c0 1 --2- 10.20.193.27:0/231659655 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f78c4115440 0x7f78c411d720 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:02.219 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.214+0000 7f78cade76c0 1 -- 10.20.193.27:0/231659655 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f78c4115440 msgr2=0x7f78c411d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:02.219 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.214+0000 7f78cade76c0 1 --2- 10.20.193.27:0/231659655 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f78c4115440 0x7f78c411d720 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:02.219 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.214+0000 7f78cade76c0 1 -- 10.20.193.27:0/231659655 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f78c410f230 msgr2=0x7f78c411d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:02.219 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.214+0000 7f78cade76c0 1 --2- 10.20.193.27:0/231659655 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f78c410f230 0x7f78c411d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:02.219 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.214+0000 7f78cade76c0 1 -- 10.20.193.27:0/231659655 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b800a660 con 0x7f78c4122ae0 2026-02-13T02:23:02.219 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.214+0000 7f78cade76c0 1 --2- 10.20.193.27:0/231659655 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f78c4122ae0 0x7f78c4122f80 secure :-1 s=READY pgs=353 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f78b80052a0 tx=0x7f78b80052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:02.220 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.215+0000 7f78af7fe6c0 1 -- 10.20.193.27:0/231659655 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f78b8003a30 con 0x7f78c4122ae0 2026-02-13T02:23:02.220 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.215+0000 7f78af7fe6c0 1 -- 10.20.193.27:0/231659655 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f78b80057c0 con 0x7f78c4122ae0 2026-02-13T02:23:02.220 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.215+0000 7f78cc8756c0 1 -- 10.20.193.27:0/231659655 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c4163c70 con 0x7f78c4122ae0 2026-02-13T02:23:02.220 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.215+0000 7f78af7fe6c0 1 -- 10.20.193.27:0/231659655 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f78b8038070 con 0x7f78c4122ae0 2026-02-13T02:23:02.220 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.215+0000 7f78cc8756c0 1 -- 10.20.193.27:0/231659655 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c41640a0 con 0x7f78c4122ae0 2026-02-13T02:23:02.220 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.215+0000 7f78cc8756c0 1 -- 10.20.193.27:0/231659655 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78c4110500 con 0x7f78c4122ae0 2026-02-13T02:23:02.221 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.216+0000 7f78af7fe6c0 1 -- 10.20.193.27:0/231659655 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f78b8003b90 con 0x7f78c4122ae0 2026-02-13T02:23:02.221 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.217+0000 7f78af7fe6c0 1 --2- 10.20.193.27:0/231659655 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f78a00b5710 0x7f78a00d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:02.221 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.217+0000 7f78ca5e66c0 1 --2- 10.20.193.27:0/231659655 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f78a00b5710 0x7f78a00d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:02.221 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.217+0000 7f78af7fe6c0 1 -- 10.20.193.27:0/231659655 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f78b8108eb0 con 0x7f78c4122ae0 2026-02-13T02:23:02.221 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.217+0000 7f78af7fe6c0 1 -- 10.20.193.27:0/231659655 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f78b80cf230 con 0x7f78c4122ae0 2026-02-13T02:23:02.221 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.217+0000 7f78ca5e66c0 1 --2- 10.20.193.27:0/231659655 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f78a00b5710 0x7f78a00d5bc0 secure :-1 s=READY pgs=240 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f78bc004750 tx=0x7f78bc009260 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:02.332 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.331+0000 7f78cc8756c0 1 -- 10.20.193.27:0/231659655 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f78c410f600 con 0x7f78c4122ae0 2026-02-13T02:23:02.333 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:02 trial027 ceph-mon[24567]: pgmap v208: 97 pgs: 2 peering, 95 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 639 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2026-02-13T02:23:02.333 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.331+0000 7f78af7fe6c0 1 -- 10.20.193.27:0/231659655 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f78b80ce980 con 0x7f78c4122ae0 2026-02-13T02:23:02.334 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:23:02.334 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:23:02.334 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.332+0000 7f78cc8756c0 1 -- 10.20.193.27:0/231659655 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f78a00b5710 msgr2=0x7f78a00d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:02.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.332+0000 7f78cc8756c0 1 --2- 10.20.193.27:0/231659655 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f78a00b5710 0x7f78a00d5bc0 secure :-1 s=READY pgs=240 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f78bc004750 tx=0x7f78bc009260 comp rx=0 tx=0).stop 2026-02-13T02:23:02.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.332+0000 7f78cc8756c0 1 -- 10.20.193.27:0/231659655 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f78c4122ae0 msgr2=0x7f78c4122f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:02.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.332+0000 7f78cc8756c0 1 --2- 10.20.193.27:0/231659655 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f78c4122ae0 0x7f78c4122f80 secure :-1 s=READY pgs=353 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f78b80052a0 tx=0x7f78b80052d0 comp rx=0 tx=0).stop 2026-02-13T02:23:02.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.333+0000 7f78cc8756c0 1 -- 10.20.193.27:0/231659655 shutdown_connections 2026-02-13T02:23:02.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.333+0000 7f78cc8756c0 1 --2- 10.20.193.27:0/231659655 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f78a00b5710 0x7f78a00d5bc0 unknown :-1 s=CLOSED pgs=240 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:02.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.333+0000 7f78cc8756c0 1 --2- 10.20.193.27:0/231659655 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f78c4122ae0 0x7f78c4122f80 unknown :-1 s=CLOSED pgs=353 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:02.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.333+0000 7f78cc8756c0 1 --2- 10.20.193.27:0/231659655 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f78c4115440 0x7f78c411d720 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:02.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.333+0000 7f78cc8756c0 1 --2- 10.20.193.27:0/231659655 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f78c410f230 0x7f78c411d1e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:02.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.333+0000 7f78cc8756c0 1 -- 10.20.193.27:0/231659655 >> 10.20.193.27:0/231659655 conn(0x7f78c40ecb30 msgr2=0x7f78c410cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:02.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.333+0000 7f78cc8756c0 1 -- 10.20.193.27:0/231659655 shutdown_connections 2026-02-13T02:23:02.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:02.333+0000 7f78cc8756c0 1 -- 10.20.193.27:0/231659655 wait complete. 2026-02-13T02:23:02.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:02 trial028 ceph-mon[26701]: pgmap v208: 97 pgs: 2 peering, 95 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 639 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2026-02-13T02:23:02.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:02 trial107 ceph-mon[27422]: pgmap v208: 97 pgs: 2 peering, 95 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 639 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2026-02-13T02:23:03.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:03 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/231659655' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:03.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:03 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/231659655' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:03.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:03 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/231659655' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:04.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:04 trial027 ceph-mon[24567]: pgmap v209: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2026-02-13T02:23:04.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:04 trial028 ceph-mon[26701]: pgmap v209: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2026-02-13T02:23:04.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:04 trial107 ceph-mon[27422]: pgmap v209: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2026-02-13T02:23:05.377 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:23:05.495 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:23:05.618 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.615+0000 7f9feab036c0 1 -- 10.20.193.27:0/3778376537 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9fe4115990 msgr2=0x7f9fe4115d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:05.619 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.615+0000 7f9feab036c0 1 --2- 10.20.193.27:0/3778376537 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9fe4115990 0x7f9fe4115d60 secure :-1 s=READY pgs=354 gs=1 cs=0 l=1 c_cookie=12435727052410197603 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9fd40099a0 tx=0x7f9fd402ed20 comp rx=0 tx=0).stop 2026-02-13T02:23:05.619 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.615+0000 7f9feab036c0 1 -- 10.20.193.27:0/3778376537 shutdown_connections 2026-02-13T02:23:05.619 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.615+0000 7f9feab036c0 1 --2- 10.20.193.27:0/3778376537 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9fe411bba0 0x7f9fe417c540 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:05.619 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.615+0000 7f9feab036c0 1 --2- 10.20.193.27:0/3778376537 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9fe4116380 0x7f9fe411b3c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:05.619 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.615+0000 7f9feab036c0 1 --2- 10.20.193.27:0/3778376537 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9fe4115990 0x7f9fe4115d60 unknown :-1 s=CLOSED pgs=354 gs=1 cs=0 l=1 c_cookie=12435727052410197603 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:05.619 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.615+0000 7f9feab036c0 1 -- 10.20.193.27:0/3778376537 >> 10.20.193.27:0/3778376537 conn(0x7f9fe405e060 msgr2=0x7f9fe405e460 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:05.619 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.615+0000 7f9feab036c0 1 -- 10.20.193.27:0/3778376537 shutdown_connections 2026-02-13T02:23:05.619 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.615+0000 7f9feab036c0 1 -- 10.20.193.27:0/3778376537 wait complete. 2026-02-13T02:23:05.619 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.615+0000 7f9feab036c0 1 Processor -- start 2026-02-13T02:23:05.619 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.615+0000 7f9feab036c0 1 -- start start 2026-02-13T02:23:05.620 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.615+0000 7f9feab036c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9fe4115990 0x7f9fe4057c20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:05.620 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.615+0000 7f9feab036c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9fe411bba0 0x7f9fe4058160 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:05.620 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.615+0000 7f9feab036c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9fe40586a0 0x7f9fe416bd80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:05.620 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.615+0000 7f9feab036c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe416c2c0 con 0x7f9fe4115990 2026-02-13T02:23:05.620 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.615+0000 7f9feab036c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe416c430 con 0x7f9fe411bba0 2026-02-13T02:23:05.620 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.615+0000 7f9feab036c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe416c5f0 con 0x7f9fe40586a0 2026-02-13T02:23:05.620 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.615+0000 7f9fe88746c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9fe4115990 0x7f9fe4057c20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:05.620 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.615+0000 7f9fe88746c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9fe4115990 0x7f9fe4057c20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:57862/0 (socket says 10.20.193.27:57862) 2026-02-13T02:23:05.620 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.615+0000 7f9fe88746c0 1 -- 10.20.193.27:0/697506199 learned_addr learned my addr 10.20.193.27:0/697506199 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:23:05.620 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.615+0000 7f9fdbfff6c0 1 --2- 10.20.193.27:0/697506199 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9fe411bba0 0x7f9fe4058160 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:05.620 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.615+0000 7f9fe90756c0 1 --2- 10.20.193.27:0/697506199 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9fe40586a0 0x7f9fe416bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:05.620 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.615+0000 7f9fe88746c0 1 -- 10.20.193.27:0/697506199 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9fe40586a0 msgr2=0x7f9fe416bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:05.621 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.615+0000 7f9fe88746c0 1 --2- 10.20.193.27:0/697506199 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9fe40586a0 0x7f9fe416bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:05.621 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.615+0000 7f9fe88746c0 1 -- 10.20.193.27:0/697506199 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9fe411bba0 msgr2=0x7f9fe4058160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:05.621 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.615+0000 7f9fe88746c0 1 --2- 10.20.193.27:0/697506199 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9fe411bba0 0x7f9fe4058160 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:05.621 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.615+0000 7f9fe88746c0 1 -- 10.20.193.27:0/697506199 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd4009590 con 0x7f9fe4115990 2026-02-13T02:23:05.621 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.615+0000 7f9fdbfff6c0 1 --2- 10.20.193.27:0/697506199 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9fe411bba0 0x7f9fe4058160 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:23:05.621 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.615+0000 7f9fe88746c0 1 --2- 10.20.193.27:0/697506199 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9fe4115990 0x7f9fe4057c20 secure :-1 s=READY pgs=355 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9fd402f230 tx=0x7f9fd402fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:05.621 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.616+0000 7f9fd9ffb6c0 1 -- 10.20.193.27:0/697506199 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9fd4002ae0 con 0x7f9fe4115990 2026-02-13T02:23:05.621 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.616+0000 7f9fd9ffb6c0 1 -- 10.20.193.27:0/697506199 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9fd4002c40 con 0x7f9fe4115990 2026-02-13T02:23:05.621 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.616+0000 7f9feab036c0 1 -- 10.20.193.27:0/697506199 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fe416c870 con 0x7f9fe4115990 2026-02-13T02:23:05.621 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.616+0000 7f9fd9ffb6c0 1 -- 10.20.193.27:0/697506199 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9fd40372f0 con 0x7f9fe4115990 2026-02-13T02:23:05.621 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.616+0000 7f9feab036c0 1 -- 10.20.193.27:0/697506199 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fe416cd40 con 0x7f9fe4115990 2026-02-13T02:23:05.622 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.616+0000 7f9feab036c0 1 -- 10.20.193.27:0/697506199 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fe4116c60 con 0x7f9fe4115990 2026-02-13T02:23:05.622 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.617+0000 7f9fd9ffb6c0 1 -- 10.20.193.27:0/697506199 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f9fd4007810 con 0x7f9fe4115990 2026-02-13T02:23:05.623 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.618+0000 7f9fd9ffb6c0 1 --2- 10.20.193.27:0/697506199 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9fc00b5710 0x7f9fc00d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:05.623 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.618+0000 7f9fd9ffb6c0 1 -- 10.20.193.27:0/697506199 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f9fd4107d50 con 0x7f9fe4115990 2026-02-13T02:23:05.623 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.618+0000 7f9fdbfff6c0 1 --2- 10.20.193.27:0/697506199 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9fc00b5710 0x7f9fc00d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:05.623 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.618+0000 7f9fdbfff6c0 1 --2- 10.20.193.27:0/697506199 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9fc00b5710 0x7f9fc00d5bc0 secure :-1 s=READY pgs=241 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9fe4113fc0 tx=0x7f9fcc009260 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:05.623 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.618+0000 7f9fd9ffb6c0 1 -- 10.20.193.27:0/697506199 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f9fd40ca670 con 0x7f9fe4115990 2026-02-13T02:23:05.736 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.735+0000 7f9feab036c0 1 -- 10.20.193.27:0/697506199 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f9fe4058f20 con 0x7f9fe4115990 2026-02-13T02:23:05.736 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.735+0000 7f9fd9ffb6c0 1 -- 10.20.193.27:0/697506199 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f9fd40091e0 con 0x7f9fe4115990 2026-02-13T02:23:05.737 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:23:05.737 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:23:05.738 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.736+0000 7f9feab036c0 1 -- 10.20.193.27:0/697506199 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9fc00b5710 msgr2=0x7f9fc00d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:05.738 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.736+0000 7f9feab036c0 1 --2- 10.20.193.27:0/697506199 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9fc00b5710 0x7f9fc00d5bc0 secure :-1 s=READY pgs=241 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9fe4113fc0 tx=0x7f9fcc009260 comp rx=0 tx=0).stop 2026-02-13T02:23:05.738 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.736+0000 7f9feab036c0 1 -- 10.20.193.27:0/697506199 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9fe4115990 msgr2=0x7f9fe4057c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:05.738 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.736+0000 7f9feab036c0 1 --2- 10.20.193.27:0/697506199 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9fe4115990 0x7f9fe4057c20 secure :-1 s=READY pgs=355 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9fd402f230 tx=0x7f9fd402fc00 comp rx=0 tx=0).stop 2026-02-13T02:23:05.738 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.736+0000 7f9feab036c0 1 -- 10.20.193.27:0/697506199 shutdown_connections 2026-02-13T02:23:05.738 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.736+0000 7f9feab036c0 1 --2- 10.20.193.27:0/697506199 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9fc00b5710 0x7f9fc00d5bc0 unknown :-1 s=CLOSED pgs=241 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:05.738 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.736+0000 7f9feab036c0 1 --2- 10.20.193.27:0/697506199 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9fe40586a0 0x7f9fe416bd80 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:05.738 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.736+0000 7f9feab036c0 1 --2- 10.20.193.27:0/697506199 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9fe411bba0 0x7f9fe4058160 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:05.739 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.736+0000 7f9feab036c0 1 --2- 10.20.193.27:0/697506199 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9fe4115990 0x7f9fe4057c20 unknown :-1 s=CLOSED pgs=355 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:05.739 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.736+0000 7f9feab036c0 1 -- 10.20.193.27:0/697506199 >> 10.20.193.27:0/697506199 conn(0x7f9fe405e060 msgr2=0x7f9fe4118dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:05.739 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.736+0000 7f9feab036c0 1 -- 10.20.193.27:0/697506199 shutdown_connections 2026-02-13T02:23:05.739 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:05.736+0000 7f9feab036c0 1 -- 10.20.193.27:0/697506199 wait complete. 2026-02-13T02:23:06.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:06 trial027 ceph-mon[24567]: pgmap v210: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 639 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2026-02-13T02:23:06.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:06 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/697506199' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:06.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:06 trial028 ceph-mon[26701]: pgmap v210: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 639 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2026-02-13T02:23:06.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:06 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/697506199' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:06.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:06 trial107 ceph-mon[27422]: pgmap v210: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 639 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2026-02-13T02:23:06.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:06 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/697506199' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:08.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:08 trial027 ceph-mon[24567]: pgmap v211: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2026-02-13T02:23:08.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:08 trial028 ceph-mon[26701]: pgmap v211: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2026-02-13T02:23:08.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:08 trial107 ceph-mon[27422]: pgmap v211: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2026-02-13T02:23:08.780 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:23:08.900 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:23:09.022 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.018+0000 7f821ca5f6c0 1 -- 10.20.193.27:0/3566381036 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8218115410 msgr2=0x7f8218175db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:09.022 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.018+0000 7f821ca5f6c0 1 --2- 10.20.193.27:0/3566381036 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8218115410 0x7f8218175db0 secure :-1 s=READY pgs=356 gs=1 cs=0 l=1 c_cookie=4575697991083817436 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f820800aa70 tx=0x7f820802fde0 comp rx=0 tx=0).stop 2026-02-13T02:23:09.022 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.018+0000 7f821ca5f6c0 1 -- 10.20.193.27:0/3566381036 shutdown_connections 2026-02-13T02:23:09.023 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.018+0000 7f821ca5f6c0 1 --2- 10.20.193.27:0/3566381036 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8218115410 0x7f8218175db0 unknown :-1 s=CLOSED pgs=356 gs=1 cs=0 l=1 c_cookie=4575697991083817436 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:09.023 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.018+0000 7f821ca5f6c0 1 --2- 10.20.193.27:0/3566381036 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f821810fbf0 0x7f8218114c30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:09.023 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.018+0000 7f821ca5f6c0 1 --2- 10.20.193.27:0/3566381036 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f821810f200 0x7f821810f5d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:09.023 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.018+0000 7f821ca5f6c0 1 -- 10.20.193.27:0/3566381036 >> 10.20.193.27:0/3566381036 conn(0x7f82180ecb00 msgr2=0x7f821810cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:09.023 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.018+0000 7f821ca5f6c0 1 -- 10.20.193.27:0/3566381036 shutdown_connections 2026-02-13T02:23:09.023 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.018+0000 7f821ca5f6c0 1 -- 10.20.193.27:0/3566381036 wait complete. 2026-02-13T02:23:09.023 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.018+0000 7f821ca5f6c0 1 Processor -- start 2026-02-13T02:23:09.023 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.018+0000 7f821ca5f6c0 1 -- start start 2026-02-13T02:23:09.023 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.018+0000 7f821ca5f6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f821810f200 0x7f821811d250 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:09.023 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.018+0000 7f821ca5f6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8218115410 0x7f821811d790 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:09.023 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.018+0000 7f821ca5f6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8218122b50 0x7f8218122ff0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:09.024 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.018+0000 7f821ca5f6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8218123560 con 0x7f821810f200 2026-02-13T02:23:09.024 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.018+0000 7f821ca5f6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f82181236d0 con 0x7f8218115410 2026-02-13T02:23:09.024 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.018+0000 7f821ca5f6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8218123890 con 0x7f8218122b50 2026-02-13T02:23:09.024 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.018+0000 7f82165756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f821810f200 0x7f821811d250 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:09.024 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.018+0000 7f82165756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f821810f200 0x7f821811d250 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:51444/0 (socket says 10.20.193.27:51444) 2026-02-13T02:23:09.024 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.018+0000 7f82165756c0 1 -- 10.20.193.27:0/1572374251 learned_addr learned my addr 10.20.193.27:0/1572374251 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:23:09.024 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.019+0000 7f8216d766c0 1 --2- 10.20.193.27:0/1572374251 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8218122b50 0x7f8218122ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:09.024 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.019+0000 7f8215d746c0 1 --2- 10.20.193.27:0/1572374251 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8218115410 0x7f821811d790 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:09.024 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.019+0000 7f82165756c0 1 -- 10.20.193.27:0/1572374251 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8218122b50 msgr2=0x7f8218122ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:09.024 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.019+0000 7f82165756c0 1 --2- 10.20.193.27:0/1572374251 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8218122b50 0x7f8218122ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:09.024 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.019+0000 7f82165756c0 1 -- 10.20.193.27:0/1572374251 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8218115410 msgr2=0x7f821811d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:09.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.019+0000 7f82165756c0 1 --2- 10.20.193.27:0/1572374251 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8218115410 0x7f821811d790 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:09.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.019+0000 7f82165756c0 1 -- 10.20.193.27:0/1572374251 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f820800a660 con 0x7f821810f200 2026-02-13T02:23:09.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.019+0000 7f82165756c0 1 --2- 10.20.193.27:0/1572374251 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f821810f200 0x7f821811d250 secure :-1 s=READY pgs=357 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f820c0092e0 tx=0x7f820c00bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:09.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.019+0000 7f81ff7fe6c0 1 -- 10.20.193.27:0/1572374251 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f820c004250 con 0x7f821810f200 2026-02-13T02:23:09.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.019+0000 7f821ca5f6c0 1 -- 10.20.193.27:0/1572374251 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8218163d40 con 0x7f821810f200 2026-02-13T02:23:09.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.019+0000 7f81ff7fe6c0 1 -- 10.20.193.27:0/1572374251 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f820c0043b0 con 0x7f821810f200 2026-02-13T02:23:09.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.019+0000 7f81ff7fe6c0 1 -- 10.20.193.27:0/1572374251 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f820c00cd40 con 0x7f821810f200 2026-02-13T02:23:09.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.019+0000 7f821ca5f6c0 1 -- 10.20.193.27:0/1572374251 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82181641d0 con 0x7f821810f200 2026-02-13T02:23:09.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.019+0000 7f821ca5f6c0 1 -- 10.20.193.27:0/1572374251 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82181104d0 con 0x7f821810f200 2026-02-13T02:23:09.026 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.021+0000 7f81ff7fe6c0 1 -- 10.20.193.27:0/1572374251 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f820c0026b0 con 0x7f821810f200 2026-02-13T02:23:09.027 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.021+0000 7f81ff7fe6c0 1 --2- 10.20.193.27:0/1572374251 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f81f00b5690 0x7f81f00d5b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:09.027 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.021+0000 7f81ff7fe6c0 1 -- 10.20.193.27:0/1572374251 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f820c0e3bf0 con 0x7f821810f200 2026-02-13T02:23:09.027 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.021+0000 7f8215d746c0 1 --2- 10.20.193.27:0/1572374251 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f81f00b5690 0x7f81f00d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:09.027 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.022+0000 7f81ff7fe6c0 1 -- 10.20.193.27:0/1572374251 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f820c0a6550 con 0x7f821810f200 2026-02-13T02:23:09.027 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.022+0000 7f8215d746c0 1 --2- 10.20.193.27:0/1572374251 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f81f00b5690 0x7f81f00d5b40 secure :-1 s=READY pgs=242 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8218123be0 tx=0x7f82000094f0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:09.138 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.136+0000 7f821ca5f6c0 1 -- 10.20.193.27:0/1572374251 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f821810f5d0 con 0x7f821810f200 2026-02-13T02:23:09.138 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.136+0000 7f81ff7fe6c0 1 -- 10.20.193.27:0/1572374251 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f820c0a9700 con 0x7f821810f200 2026-02-13T02:23:09.139 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:23:09.139 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:23:09.140 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.137+0000 7f821ca5f6c0 1 -- 10.20.193.27:0/1572374251 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f81f00b5690 msgr2=0x7f81f00d5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:09.140 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.137+0000 7f821ca5f6c0 1 --2- 10.20.193.27:0/1572374251 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f81f00b5690 0x7f81f00d5b40 secure :-1 s=READY pgs=242 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8218123be0 tx=0x7f82000094f0 comp rx=0 tx=0).stop 2026-02-13T02:23:09.140 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.137+0000 7f821ca5f6c0 1 -- 10.20.193.27:0/1572374251 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f821810f200 msgr2=0x7f821811d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:09.140 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.137+0000 7f821ca5f6c0 1 --2- 10.20.193.27:0/1572374251 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f821810f200 0x7f821811d250 secure :-1 s=READY pgs=357 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f820c0092e0 tx=0x7f820c00bc50 comp rx=0 tx=0).stop 2026-02-13T02:23:09.140 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.137+0000 7f821ca5f6c0 1 -- 10.20.193.27:0/1572374251 shutdown_connections 2026-02-13T02:23:09.141 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.137+0000 7f821ca5f6c0 1 --2- 10.20.193.27:0/1572374251 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f81f00b5690 0x7f81f00d5b40 unknown :-1 s=CLOSED pgs=242 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:09.141 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.137+0000 7f821ca5f6c0 1 --2- 10.20.193.27:0/1572374251 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8218122b50 0x7f8218122ff0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:09.141 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.137+0000 7f821ca5f6c0 1 --2- 10.20.193.27:0/1572374251 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8218115410 0x7f821811d790 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:09.141 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.137+0000 7f821ca5f6c0 1 --2- 10.20.193.27:0/1572374251 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f821810f200 0x7f821811d250 unknown :-1 s=CLOSED pgs=357 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:09.141 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.137+0000 7f821ca5f6c0 1 -- 10.20.193.27:0/1572374251 >> 10.20.193.27:0/1572374251 conn(0x7f82180ecb00 msgr2=0x7f821810cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:09.141 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.137+0000 7f821ca5f6c0 1 -- 10.20.193.27:0/1572374251 shutdown_connections 2026-02-13T02:23:09.141 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:09.137+0000 7f821ca5f6c0 1 -- 10.20.193.27:0/1572374251 wait complete. 2026-02-13T02:23:10.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:10 trial027 ceph-mon[24567]: pgmap v212: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 941 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2026-02-13T02:23:10.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:10 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1572374251' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:10.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:10 trial028 ceph-mon[26701]: pgmap v212: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 941 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2026-02-13T02:23:10.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:10 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1572374251' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:10.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:10 trial107 ceph-mon[27422]: pgmap v212: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 941 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2026-02-13T02:23:10.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:10 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1572374251' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:12.181 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:23:12.302 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:23:12.356 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:12 trial027 ceph-mon[24567]: pgmap v213: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2026-02-13T02:23:12.416 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.411+0000 7fc0c85206c0 1 -- 10.20.193.27:0/3275809941 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc0c010fc20 msgr2=0x7fc0c0114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:12.416 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.411+0000 7fc0c85206c0 1 --2- 10.20.193.27:0/3275809941 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc0c010fc20 0x7fc0c0114c60 secure :-1 s=READY pgs=358 gs=1 cs=0 l=1 c_cookie=11330002968625310860 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc0b40099a0 tx=0x7fc0b402ed20 comp rx=0 tx=0).stop 2026-02-13T02:23:12.416 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.411+0000 7fc0c85206c0 1 -- 10.20.193.27:0/3275809941 shutdown_connections 2026-02-13T02:23:12.416 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.411+0000 7fc0c85206c0 1 --2- 10.20.193.27:0/3275809941 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc0c0115440 0x7fc0c0175de0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:12.416 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.411+0000 7fc0c85206c0 1 --2- 10.20.193.27:0/3275809941 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc0c010fc20 0x7fc0c0114c60 unknown :-1 s=CLOSED pgs=358 gs=1 cs=0 l=1 c_cookie=11330002968625310860 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:12.416 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.411+0000 7fc0c85206c0 1 --2- 10.20.193.27:0/3275809941 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc0c010f230 0x7fc0c010f600 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:12.417 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.411+0000 7fc0c85206c0 1 -- 10.20.193.27:0/3275809941 >> 10.20.193.27:0/3275809941 conn(0x7fc0c00ecb30 msgr2=0x7fc0c010cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:12.417 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.411+0000 7fc0c85206c0 1 -- 10.20.193.27:0/3275809941 shutdown_connections 2026-02-13T02:23:12.417 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.411+0000 7fc0c85206c0 1 -- 10.20.193.27:0/3275809941 wait complete. 2026-02-13T02:23:12.417 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.412+0000 7fc0c85206c0 1 Processor -- start 2026-02-13T02:23:12.417 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.412+0000 7fc0c85206c0 1 -- start start 2026-02-13T02:23:12.417 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.412+0000 7fc0c85206c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc0c010f230 0x7fc0c00f4c60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:12.417 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.412+0000 7fc0c85206c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc0c0115440 0x7fc0c0175ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:12.417 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.412+0000 7fc0c85206c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc0c00f5290 0x7fc0c00fa560 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:12.417 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.412+0000 7fc0c85206c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c00f5980 con 0x7fc0c00f5290 2026-02-13T02:23:12.417 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.412+0000 7fc0c85206c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c00faaa0 con 0x7fc0c010f230 2026-02-13T02:23:12.417 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.412+0000 7fc0c85206c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c00fac60 con 0x7fc0c0115440 2026-02-13T02:23:12.417 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.412+0000 7fc0c6a926c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc0c00f5290 0x7fc0c00fa560 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:12.418 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.412+0000 7fc0c6a926c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc0c00f5290 0x7fc0c00fa560 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:51454/0 (socket says 10.20.193.27:51454) 2026-02-13T02:23:12.418 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.412+0000 7fc0c6a926c0 1 -- 10.20.193.27:0/3451734749 learned_addr learned my addr 10.20.193.27:0/3451734749 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:23:12.418 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.412+0000 7fc0c62916c0 1 --2- 10.20.193.27:0/3451734749 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc0c010f230 0x7fc0c00f4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:12.418 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.412+0000 7fc0c5a906c0 1 --2- 10.20.193.27:0/3451734749 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc0c0115440 0x7fc0c0175ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:12.418 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.412+0000 7fc0c6a926c0 1 -- 10.20.193.27:0/3451734749 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc0c0115440 msgr2=0x7fc0c0175ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:12.418 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.412+0000 7fc0c6a926c0 1 --2- 10.20.193.27:0/3451734749 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc0c0115440 0x7fc0c0175ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:12.418 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.412+0000 7fc0c6a926c0 1 -- 10.20.193.27:0/3451734749 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc0c010f230 msgr2=0x7fc0c00f4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:12.418 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.412+0000 7fc0c6a926c0 1 --2- 10.20.193.27:0/3451734749 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc0c010f230 0x7fc0c00f4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:12.418 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.412+0000 7fc0c6a926c0 1 -- 10.20.193.27:0/3451734749 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b4009590 con 0x7fc0c00f5290 2026-02-13T02:23:12.418 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.412+0000 7fc0c6a926c0 1 --2- 10.20.193.27:0/3451734749 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc0c00f5290 0x7fc0c00fa560 secure :-1 s=READY pgs=359 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc0bc00c6c0 tx=0x7fc0bc00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:12.418 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.412+0000 7fc0a77fe6c0 1 -- 10.20.193.27:0/3451734749 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc0bc003a30 con 0x7fc0c00f5290 2026-02-13T02:23:12.418 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.412+0000 7fc0a77fe6c0 1 -- 10.20.193.27:0/3451734749 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc0bc0054e0 con 0x7fc0c00f5290 2026-02-13T02:23:12.419 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.412+0000 7fc0a77fe6c0 1 -- 10.20.193.27:0/3451734749 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc0bc00dd10 con 0x7fc0c00f5290 2026-02-13T02:23:12.419 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.412+0000 7fc0c85206c0 1 -- 10.20.193.27:0/3451734749 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0c00faf40 con 0x7fc0c00f5290 2026-02-13T02:23:12.419 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.412+0000 7fc0c85206c0 1 -- 10.20.193.27:0/3451734749 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0c00fb6d0 con 0x7fc0c00f5290 2026-02-13T02:23:12.419 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.413+0000 7fc0c85206c0 1 -- 10.20.193.27:0/3451734749 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0c0110500 con 0x7fc0c00f5290 2026-02-13T02:23:12.420 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.414+0000 7fc0a77fe6c0 1 -- 10.20.193.27:0/3451734749 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7fc0bc005080 con 0x7fc0c00f5290 2026-02-13T02:23:12.420 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.415+0000 7fc0a77fe6c0 1 --2- 10.20.193.27:0/3451734749 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc09c0b5710 0x7fc09c0d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:12.420 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.415+0000 7fc0a77fe6c0 1 -- 10.20.193.27:0/3451734749 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7fc0bc0e4950 con 0x7fc0c00f5290 2026-02-13T02:23:12.420 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.415+0000 7fc0c62916c0 1 --2- 10.20.193.27:0/3451734749 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc09c0b5710 0x7fc09c0d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:12.420 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.415+0000 7fc0a77fe6c0 1 -- 10.20.193.27:0/3451734749 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fc0bc0a7270 con 0x7fc0c00f5290 2026-02-13T02:23:12.420 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.415+0000 7fc0c62916c0 1 --2- 10.20.193.27:0/3451734749 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc09c0b5710 0x7fc09c0d5bc0 secure :-1 s=READY pgs=243 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc0b0004750 tx=0x7fc0b0009190 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:12.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:12 trial028 ceph-mon[26701]: pgmap v213: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2026-02-13T02:23:12.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:12 trial107 ceph-mon[27422]: pgmap v213: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2026-02-13T02:23:12.532 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.530+0000 7fc0c85206c0 1 -- 10.20.193.27:0/3451734749 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc0c010f600 con 0x7fc0c00f5290 2026-02-13T02:23:12.532 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.530+0000 7fc0a77fe6c0 1 -- 10.20.193.27:0/3451734749 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7fc0bc0aa420 con 0x7fc0c00f5290 2026-02-13T02:23:12.532 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:23:12.533 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:23:12.534 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.531+0000 7fc0c85206c0 1 -- 10.20.193.27:0/3451734749 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc09c0b5710 msgr2=0x7fc09c0d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:12.534 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.531+0000 7fc0c85206c0 1 --2- 10.20.193.27:0/3451734749 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc09c0b5710 0x7fc09c0d5bc0 secure :-1 s=READY pgs=243 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc0b0004750 tx=0x7fc0b0009190 comp rx=0 tx=0).stop 2026-02-13T02:23:12.534 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.531+0000 7fc0c85206c0 1 -- 10.20.193.27:0/3451734749 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc0c00f5290 msgr2=0x7fc0c00fa560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:12.534 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.531+0000 7fc0c85206c0 1 --2- 10.20.193.27:0/3451734749 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc0c00f5290 0x7fc0c00fa560 secure :-1 s=READY pgs=359 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc0bc00c6c0 tx=0x7fc0bc00cb90 comp rx=0 tx=0).stop 2026-02-13T02:23:12.534 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.531+0000 7fc0c85206c0 1 -- 10.20.193.27:0/3451734749 shutdown_connections 2026-02-13T02:23:12.534 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.531+0000 7fc0c85206c0 1 --2- 10.20.193.27:0/3451734749 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc09c0b5710 0x7fc09c0d5bc0 unknown :-1 s=CLOSED pgs=243 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:12.534 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.531+0000 7fc0c85206c0 1 --2- 10.20.193.27:0/3451734749 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc0c00f5290 0x7fc0c00fa560 unknown :-1 s=CLOSED pgs=359 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:12.534 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.531+0000 7fc0c85206c0 1 --2- 10.20.193.27:0/3451734749 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc0c0115440 0x7fc0c0175ce0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:12.534 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.531+0000 7fc0c85206c0 1 --2- 10.20.193.27:0/3451734749 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc0c010f230 0x7fc0c00f4c60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:12.534 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.531+0000 7fc0c85206c0 1 -- 10.20.193.27:0/3451734749 >> 10.20.193.27:0/3451734749 conn(0x7fc0c00ecb30 msgr2=0x7fc0c010d930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:12.535 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.531+0000 7fc0c85206c0 1 -- 10.20.193.27:0/3451734749 shutdown_connections 2026-02-13T02:23:12.535 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:12.531+0000 7fc0c85206c0 1 -- 10.20.193.27:0/3451734749 wait complete. 2026-02-13T02:23:13.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:13 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3451734749' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:13.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:13 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3451734749' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:13.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:13 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3451734749' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:14.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:14 trial027 ceph-mon[24567]: pgmap v214: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:23:14.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:14 trial028 ceph-mon[26701]: pgmap v214: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:23:14.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:14 trial107 ceph-mon[27422]: pgmap v214: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:23:15.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:15 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:23:15.423 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:15 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:23:15.425 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:15 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:23:15.574 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:23:15.696 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:23:15.817 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.813+0000 7f43ea9d76c0 1 -- 10.20.193.27:0/295627033 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f43e410cf80 msgr2=0x7f43e410d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:15.817 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.813+0000 7f43ea9d76c0 1 --2- 10.20.193.27:0/295627033 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f43e410cf80 0x7f43e410d350 secure :-1 s=READY pgs=360 gs=1 cs=0 l=1 c_cookie=5858423217629197411 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f43d00099a0 tx=0x7f43d002ed20 comp rx=0 tx=0).stop 2026-02-13T02:23:15.818 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.813+0000 7f43ea9d76c0 1 -- 10.20.193.27:0/295627033 shutdown_connections 2026-02-13T02:23:15.818 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.813+0000 7f43ea9d76c0 1 --2- 10.20.193.27:0/295627033 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f43e4113190 0x7f43e4173b30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:15.818 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.813+0000 7f43ea9d76c0 1 --2- 10.20.193.27:0/295627033 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f43e410d970 0x7f43e41129b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:15.818 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.813+0000 7f43ea9d76c0 1 --2- 10.20.193.27:0/295627033 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f43e410cf80 0x7f43e410d350 unknown :-1 s=CLOSED pgs=360 gs=1 cs=0 l=1 c_cookie=5858423217629197411 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:15.818 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.813+0000 7f43ea9d76c0 1 -- 10.20.193.27:0/295627033 >> 10.20.193.27:0/295627033 conn(0x7f43e40ea880 msgr2=0x7f43e410ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:15.818 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.813+0000 7f43ea9d76c0 1 -- 10.20.193.27:0/295627033 shutdown_connections 2026-02-13T02:23:15.818 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.813+0000 7f43ea9d76c0 1 -- 10.20.193.27:0/295627033 wait complete. 2026-02-13T02:23:15.818 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.813+0000 7f43ea9d76c0 1 Processor -- start 2026-02-13T02:23:15.818 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.813+0000 7f43ea9d76c0 1 -- start start 2026-02-13T02:23:15.818 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.813+0000 7f43ea9d76c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f43e410cf80 0x7f43e40f4c70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:15.819 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.813+0000 7f43ea9d76c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f43e4113190 0x7f43e4173a30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:15.819 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.813+0000 7f43ea9d76c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f43e40f52a0 0x7f43e40fa570 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:15.819 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.813+0000 7f43ea9d76c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f43e40f5990 con 0x7f43e40f52a0 2026-02-13T02:23:15.819 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.813+0000 7f43ea9d76c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f43e40faab0 con 0x7f43e410cf80 2026-02-13T02:23:15.819 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.813+0000 7f43ea9d76c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f43e40fac70 con 0x7f43e4113190 2026-02-13T02:23:15.819 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.813+0000 7f43e8f496c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f43e40f52a0 0x7f43e40fa570 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:15.819 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.813+0000 7f43e8f496c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f43e40f52a0 0x7f43e40fa570 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:51476/0 (socket says 10.20.193.27:51476) 2026-02-13T02:23:15.819 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.813+0000 7f43e8f496c0 1 -- 10.20.193.27:0/1496060659 learned_addr learned my addr 10.20.193.27:0/1496060659 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:23:15.819 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.813+0000 7f43e3fff6c0 1 --2- 10.20.193.27:0/1496060659 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f43e410cf80 0x7f43e40f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:15.819 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.813+0000 7f43e37fe6c0 1 --2- 10.20.193.27:0/1496060659 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f43e4113190 0x7f43e4173a30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:15.819 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.813+0000 7f43e8f496c0 1 -- 10.20.193.27:0/1496060659 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f43e4113190 msgr2=0x7f43e4173a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:15.819 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.813+0000 7f43e8f496c0 1 --2- 10.20.193.27:0/1496060659 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f43e4113190 0x7f43e4173a30 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:15.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.813+0000 7f43e8f496c0 1 -- 10.20.193.27:0/1496060659 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f43e410cf80 msgr2=0x7f43e40f4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:15.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.814+0000 7f43e8f496c0 1 --2- 10.20.193.27:0/1496060659 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f43e410cf80 0x7f43e40f4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:15.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.814+0000 7f43e8f496c0 1 -- 10.20.193.27:0/1496060659 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d0009590 con 0x7f43e40f52a0 2026-02-13T02:23:15.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.814+0000 7f43e37fe6c0 1 --2- 10.20.193.27:0/1496060659 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f43e4113190 0x7f43e4173a30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:23:15.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.814+0000 7f43e8f496c0 1 --2- 10.20.193.27:0/1496060659 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f43e40f52a0 0x7f43e40fa570 secure :-1 s=READY pgs=361 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f43d8006ea0 tx=0x7f43d800ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:15.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.814+0000 7f43e17fa6c0 1 -- 10.20.193.27:0/1496060659 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f43d8003c40 con 0x7f43e40f52a0 2026-02-13T02:23:15.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.814+0000 7f43ea9d76c0 1 -- 10.20.193.27:0/1496060659 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43e40faf50 con 0x7f43e40f52a0 2026-02-13T02:23:15.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.814+0000 7f43e17fa6c0 1 -- 10.20.193.27:0/1496060659 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f43d8003da0 con 0x7f43e40f52a0 2026-02-13T02:23:15.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.814+0000 7f43e17fa6c0 1 -- 10.20.193.27:0/1496060659 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f43d80081c0 con 0x7f43e40f52a0 2026-02-13T02:23:15.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.814+0000 7f43ea9d76c0 1 -- 10.20.193.27:0/1496060659 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43e40fb6e0 con 0x7f43e40f52a0 2026-02-13T02:23:15.820 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.814+0000 7f43ea9d76c0 1 -- 10.20.193.27:0/1496060659 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43e410e250 con 0x7f43e40f52a0 2026-02-13T02:23:15.821 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.815+0000 7f43e17fa6c0 1 -- 10.20.193.27:0/1496060659 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f43d8005860 con 0x7f43e40f52a0 2026-02-13T02:23:15.822 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.816+0000 7f43e17fa6c0 1 --2- 10.20.193.27:0/1496060659 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f43b80b5690 0x7f43b80d5b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:15.822 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.816+0000 7f43e3fff6c0 1 --2- 10.20.193.27:0/1496060659 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f43b80b5690 0x7f43b80d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:15.822 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.816+0000 7f43e17fa6c0 1 -- 10.20.193.27:0/1496060659 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f43d80e4c50 con 0x7f43e40f52a0 2026-02-13T02:23:15.822 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.816+0000 7f43e3fff6c0 1 --2- 10.20.193.27:0/1496060659 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f43b80b5690 0x7f43b80d5b40 secure :-1 s=READY pgs=244 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f43d00023e0 tx=0x7f43d0005de0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:15.822 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.817+0000 7f43e17fa6c0 1 -- 10.20.193.27:0/1496060659 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f43d80a7570 con 0x7f43e40f52a0 2026-02-13T02:23:15.933 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.930+0000 7f43ea9d76c0 1 -- 10.20.193.27:0/1496060659 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f43e40f5dc0 con 0x7f43e40f52a0 2026-02-13T02:23:15.933 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.931+0000 7f43e17fa6c0 1 -- 10.20.193.27:0/1496060659 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f43d80aa720 con 0x7f43e40f52a0 2026-02-13T02:23:15.934 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:23:15.934 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:23:15.935 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.932+0000 7f43ea9d76c0 1 -- 10.20.193.27:0/1496060659 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f43b80b5690 msgr2=0x7f43b80d5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:15.935 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.932+0000 7f43ea9d76c0 1 --2- 10.20.193.27:0/1496060659 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f43b80b5690 0x7f43b80d5b40 secure :-1 s=READY pgs=244 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f43d00023e0 tx=0x7f43d0005de0 comp rx=0 tx=0).stop 2026-02-13T02:23:15.935 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.932+0000 7f43ea9d76c0 1 -- 10.20.193.27:0/1496060659 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f43e40f52a0 msgr2=0x7f43e40fa570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:15.935 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.932+0000 7f43ea9d76c0 1 --2- 10.20.193.27:0/1496060659 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f43e40f52a0 0x7f43e40fa570 secure :-1 s=READY pgs=361 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f43d8006ea0 tx=0x7f43d800ce50 comp rx=0 tx=0).stop 2026-02-13T02:23:15.935 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.932+0000 7f43ea9d76c0 1 -- 10.20.193.27:0/1496060659 shutdown_connections 2026-02-13T02:23:15.935 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.932+0000 7f43ea9d76c0 1 --2- 10.20.193.27:0/1496060659 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f43b80b5690 0x7f43b80d5b40 unknown :-1 s=CLOSED pgs=244 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:15.935 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.932+0000 7f43ea9d76c0 1 --2- 10.20.193.27:0/1496060659 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f43e40f52a0 0x7f43e40fa570 unknown :-1 s=CLOSED pgs=361 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:15.935 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.932+0000 7f43ea9d76c0 1 --2- 10.20.193.27:0/1496060659 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f43e4113190 0x7f43e4173a30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:15.935 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.932+0000 7f43ea9d76c0 1 --2- 10.20.193.27:0/1496060659 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f43e410cf80 0x7f43e40f4c70 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:15.936 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.932+0000 7f43ea9d76c0 1 -- 10.20.193.27:0/1496060659 >> 10.20.193.27:0/1496060659 conn(0x7f43e40ea880 msgr2=0x7f43e410ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:15.936 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.932+0000 7f43ea9d76c0 1 -- 10.20.193.27:0/1496060659 shutdown_connections 2026-02-13T02:23:15.936 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:15.932+0000 7f43ea9d76c0 1 -- 10.20.193.27:0/1496060659 wait complete. 2026-02-13T02:23:16.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:16 trial027 ceph-mon[24567]: pgmap v215: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:23:16.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:16 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:23:16.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:16 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:23:16.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:16 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:23:16.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:16 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:23:16.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:16 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:23:16.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:16 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:23:16.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:16 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1496060659' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:16.445 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:16 trial107 ceph-mon[27422]: pgmap v215: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:23:16.445 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:16 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:23:16.446 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:16 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:23:16.446 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:16 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:23:16.446 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:16 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:23:16.446 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:16 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:23:16.446 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:16 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:23:16.446 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:16 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1496060659' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:16.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:16 trial028 ceph-mon[26701]: pgmap v215: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:23:16.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:16 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:23:16.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:16 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:23:16.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:16 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:23:16.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:16 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:23:16.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:16 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:23:16.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:16 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:23:16.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:16 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1496060659' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:17.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:17 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:23:17.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:17 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:23:17.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:17 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:23:17.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:17 trial027 ceph-mon[24567]: pgmap v216: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 915 B/s rd, 1 op/s 2026-02-13T02:23:17.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:17 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:23:17.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:17 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:23:17.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:17 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:23:17.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:17 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:23:17.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:17 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:23:17.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:17 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:23:17.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:17 trial028 ceph-mon[26701]: pgmap v216: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 915 B/s rd, 1 op/s 2026-02-13T02:23:17.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:17 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:23:17.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:17 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:23:17.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:17 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:23:17.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:17 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:23:17.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:17 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:23:17.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:17 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:23:17.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:17 trial107 ceph-mon[27422]: pgmap v216: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 915 B/s rd, 1 op/s 2026-02-13T02:23:17.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:17 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:23:17.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:17 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:23:17.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:17 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:23:18.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:18 trial027 ceph-mon[24567]: pgmap v217: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 2 op/s 2026-02-13T02:23:18.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:18 trial028 ceph-mon[26701]: pgmap v217: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 2 op/s 2026-02-13T02:23:18.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:18 trial107 ceph-mon[27422]: pgmap v217: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 2 op/s 2026-02-13T02:23:18.976 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:23:19.095 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:23:19.207 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.203+0000 7f560eee36c0 1 -- 10.20.193.27:0/329280490 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f560810fbf0 msgr2=0x7f5608114c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:19.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.203+0000 7f560eee36c0 1 --2- 10.20.193.27:0/329280490 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f560810fbf0 0x7f5608114c30 secure :-1 s=READY pgs=362 gs=1 cs=0 l=1 c_cookie=9692247108787004944 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f55f00099a0 tx=0x7f55f002ed20 comp rx=0 tx=0).stop 2026-02-13T02:23:19.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.203+0000 7f560eee36c0 1 -- 10.20.193.27:0/329280490 shutdown_connections 2026-02-13T02:23:19.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.203+0000 7f560eee36c0 1 --2- 10.20.193.27:0/329280490 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5608115410 0x7f5608175db0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:19.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.203+0000 7f560eee36c0 1 --2- 10.20.193.27:0/329280490 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f560810fbf0 0x7f5608114c30 unknown :-1 s=CLOSED pgs=362 gs=1 cs=0 l=1 c_cookie=9692247108787004944 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:19.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.203+0000 7f560eee36c0 1 --2- 10.20.193.27:0/329280490 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f560810f200 0x7f560810f5d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:19.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.203+0000 7f560eee36c0 1 -- 10.20.193.27:0/329280490 >> 10.20.193.27:0/329280490 conn(0x7f56080ecb00 msgr2=0x7f560810cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:19.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.203+0000 7f560eee36c0 1 -- 10.20.193.27:0/329280490 shutdown_connections 2026-02-13T02:23:19.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.203+0000 7f560eee36c0 1 -- 10.20.193.27:0/329280490 wait complete. 2026-02-13T02:23:19.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.204+0000 7f560eee36c0 1 Processor -- start 2026-02-13T02:23:19.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.204+0000 7f560eee36c0 1 -- start start 2026-02-13T02:23:19.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.204+0000 7f560eee36c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f560810f200 0x7f56080f4ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:19.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.204+0000 7f560eee36c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5608115410 0x7f5608175cb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:19.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.204+0000 7f560eee36c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f56080f5310 0x7f56080fa5e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:19.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.204+0000 7f560eee36c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f56080f5a00 con 0x7f560810f200 2026-02-13T02:23:19.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.204+0000 7f560eee36c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f56080fab20 con 0x7f56080f5310 2026-02-13T02:23:19.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.204+0000 7f560eee36c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f56080face0 con 0x7f5608115410 2026-02-13T02:23:19.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.204+0000 7f560cc546c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f560810f200 0x7f56080f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:19.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.204+0000 7f560cc546c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f560810f200 0x7f56080f4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:46794/0 (socket says 10.20.193.27:46794) 2026-02-13T02:23:19.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.204+0000 7f560cc546c0 1 -- 10.20.193.27:0/344700998 learned_addr learned my addr 10.20.193.27:0/344700998 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:23:19.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.204+0000 7f55fffff6c0 1 --2- 10.20.193.27:0/344700998 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5608115410 0x7f5608175cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:19.209 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.204+0000 7f560d4556c0 1 --2- 10.20.193.27:0/344700998 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f56080f5310 0x7f56080fa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:19.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.204+0000 7f560cc546c0 1 -- 10.20.193.27:0/344700998 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5608115410 msgr2=0x7f5608175cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:19.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.204+0000 7f560cc546c0 1 --2- 10.20.193.27:0/344700998 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5608115410 0x7f5608175cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:19.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.204+0000 7f560cc546c0 1 -- 10.20.193.27:0/344700998 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f56080f5310 msgr2=0x7f56080fa5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:19.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.204+0000 7f560cc546c0 1 --2- 10.20.193.27:0/344700998 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f56080f5310 0x7f56080fa5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:19.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.204+0000 7f560cc546c0 1 -- 10.20.193.27:0/344700998 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f0009590 con 0x7f560810f200 2026-02-13T02:23:19.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.204+0000 7f560d4556c0 1 --2- 10.20.193.27:0/344700998 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f56080f5310 0x7f56080fa5e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:23:19.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.204+0000 7f560cc546c0 1 --2- 10.20.193.27:0/344700998 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f560810f200 0x7f56080f4ce0 secure :-1 s=READY pgs=363 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f55f800b710 tx=0x7f55f800bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:19.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.204+0000 7f55fdffb6c0 1 -- 10.20.193.27:0/344700998 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f55f8004250 con 0x7f560810f200 2026-02-13T02:23:19.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.204+0000 7f560eee36c0 1 -- 10.20.193.27:0/344700998 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56080fafc0 con 0x7f560810f200 2026-02-13T02:23:19.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.204+0000 7f55fdffb6c0 1 -- 10.20.193.27:0/344700998 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f55f8002d80 con 0x7f560810f200 2026-02-13T02:23:19.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.204+0000 7f55fdffb6c0 1 -- 10.20.193.27:0/344700998 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f55f800cda0 con 0x7f560810f200 2026-02-13T02:23:19.210 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.204+0000 7f560eee36c0 1 -- 10.20.193.27:0/344700998 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56080fb750 con 0x7f560810f200 2026-02-13T02:23:19.211 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.205+0000 7f560eee36c0 1 -- 10.20.193.27:0/344700998 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56081104d0 con 0x7f560810f200 2026-02-13T02:23:19.212 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.206+0000 7f55fdffb6c0 1 -- 10.20.193.27:0/344700998 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f55f80028c0 con 0x7f560810f200 2026-02-13T02:23:19.212 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.207+0000 7f55fdffb6c0 1 --2- 10.20.193.27:0/344700998 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f55e40b5690 0x7f55e40d5b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:19.212 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.207+0000 7f55fffff6c0 1 --2- 10.20.193.27:0/344700998 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f55e40b5690 0x7f55e40d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:19.212 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.207+0000 7f55fdffb6c0 1 -- 10.20.193.27:0/344700998 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f55f80e3cc0 con 0x7f560810f200 2026-02-13T02:23:19.212 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.207+0000 7f55fffff6c0 1 --2- 10.20.193.27:0/344700998 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f55e40b5690 0x7f55e40d5b40 secure :-1 s=READY pgs=245 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f55f0005b40 tx=0x7f55f00092a0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:19.212 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.207+0000 7f55fdffb6c0 1 -- 10.20.193.27:0/344700998 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f55f80a6620 con 0x7f560810f200 2026-02-13T02:23:19.331 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.328+0000 7f560eee36c0 1 -- 10.20.193.27:0/344700998 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f56080f5e30 con 0x7f560810f200 2026-02-13T02:23:19.331 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.329+0000 7f55fdffb6c0 1 -- 10.20.193.27:0/344700998 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f55f80a97d0 con 0x7f560810f200 2026-02-13T02:23:19.331 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:23:19.331 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:23:19.333 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.330+0000 7f560eee36c0 1 -- 10.20.193.27:0/344700998 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f55e40b5690 msgr2=0x7f55e40d5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:19.333 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.330+0000 7f560eee36c0 1 --2- 10.20.193.27:0/344700998 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f55e40b5690 0x7f55e40d5b40 secure :-1 s=READY pgs=245 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f55f0005b40 tx=0x7f55f00092a0 comp rx=0 tx=0).stop 2026-02-13T02:23:19.333 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.330+0000 7f560eee36c0 1 -- 10.20.193.27:0/344700998 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f560810f200 msgr2=0x7f56080f4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:19.333 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.330+0000 7f560eee36c0 1 --2- 10.20.193.27:0/344700998 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f560810f200 0x7f56080f4ce0 secure :-1 s=READY pgs=363 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f55f800b710 tx=0x7f55f800bbe0 comp rx=0 tx=0).stop 2026-02-13T02:23:19.333 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.330+0000 7f560eee36c0 1 -- 10.20.193.27:0/344700998 shutdown_connections 2026-02-13T02:23:19.333 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.330+0000 7f560eee36c0 1 --2- 10.20.193.27:0/344700998 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f55e40b5690 0x7f55e40d5b40 unknown :-1 s=CLOSED pgs=245 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:19.333 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.330+0000 7f560eee36c0 1 --2- 10.20.193.27:0/344700998 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f56080f5310 0x7f56080fa5e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:19.333 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.330+0000 7f560eee36c0 1 --2- 10.20.193.27:0/344700998 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5608115410 0x7f5608175cb0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:19.334 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.330+0000 7f560eee36c0 1 --2- 10.20.193.27:0/344700998 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f560810f200 0x7f56080f4ce0 unknown :-1 s=CLOSED pgs=363 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:19.334 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.330+0000 7f560eee36c0 1 -- 10.20.193.27:0/344700998 >> 10.20.193.27:0/344700998 conn(0x7f56080ecb00 msgr2=0x7f560810d900 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:19.334 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.330+0000 7f560eee36c0 1 -- 10.20.193.27:0/344700998 shutdown_connections 2026-02-13T02:23:19.334 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:19.330+0000 7f560eee36c0 1 -- 10.20.193.27:0/344700998 wait complete. 2026-02-13T02:23:20.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:20 trial027 ceph-mon[24567]: pgmap v218: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 2 op/s 2026-02-13T02:23:20.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:20 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/344700998' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:20.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:20 trial028 ceph-mon[26701]: pgmap v218: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 2 op/s 2026-02-13T02:23:20.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:20 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/344700998' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:20.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:20 trial107 ceph-mon[27422]: pgmap v218: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 2 op/s 2026-02-13T02:23:20.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:20 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/344700998' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:22.374 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:23:22.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:22 trial028 ceph-mon[26701]: pgmap v219: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 2 op/s 2026-02-13T02:23:22.494 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:23:22.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:22 trial107 ceph-mon[27422]: pgmap v219: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 2 op/s 2026-02-13T02:23:22.516 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:22 trial027 ceph-mon[24567]: pgmap v219: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 2 op/s 2026-02-13T02:23:22.608 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.603+0000 7f0033dff6c0 1 -- 10.20.193.27:0/2248786269 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f002c115440 msgr2=0x7f002c175de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:22.608 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.603+0000 7f0033dff6c0 1 --2- 10.20.193.27:0/2248786269 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f002c115440 0x7f002c175de0 secure :-1 s=READY pgs=364 gs=1 cs=0 l=1 c_cookie=871867280260132808 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f002800aa70 tx=0x7f002802fde0 comp rx=0 tx=0).stop 2026-02-13T02:23:22.608 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.603+0000 7f0033dff6c0 1 -- 10.20.193.27:0/2248786269 shutdown_connections 2026-02-13T02:23:22.608 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.603+0000 7f0033dff6c0 1 --2- 10.20.193.27:0/2248786269 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f002c115440 0x7f002c175de0 unknown :-1 s=CLOSED pgs=364 gs=1 cs=0 l=1 c_cookie=871867280260132808 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:22.608 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.603+0000 7f0033dff6c0 1 --2- 10.20.193.27:0/2248786269 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f002c10fc20 0x7f002c114c60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:22.609 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.603+0000 7f0033dff6c0 1 --2- 10.20.193.27:0/2248786269 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f002c10f230 0x7f002c10f600 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:22.609 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.603+0000 7f0033dff6c0 1 -- 10.20.193.27:0/2248786269 >> 10.20.193.27:0/2248786269 conn(0x7f002c0ecb30 msgr2=0x7f002c10cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:22.609 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.603+0000 7f0033dff6c0 1 -- 10.20.193.27:0/2248786269 shutdown_connections 2026-02-13T02:23:22.609 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.603+0000 7f0033dff6c0 1 -- 10.20.193.27:0/2248786269 wait complete. 2026-02-13T02:23:22.609 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.604+0000 7f0033dff6c0 1 Processor -- start 2026-02-13T02:23:22.609 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.604+0000 7f0033dff6c0 1 -- start start 2026-02-13T02:23:22.609 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.604+0000 7f0033dff6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f002c10f230 0x7f002c11d230 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:22.609 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.604+0000 7f0033dff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f002c115440 0x7f002c11d770 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:22.609 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.604+0000 7f0033dff6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f002c122b30 0x7f002c122fd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:22.609 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.604+0000 7f0033dff6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f002c123540 con 0x7f002c115440 2026-02-13T02:23:22.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.604+0000 7f0033dff6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f002c1236b0 con 0x7f002c10f230 2026-02-13T02:23:22.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.604+0000 7f0033dff6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f002c123870 con 0x7f002c122b30 2026-02-13T02:23:22.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.604+0000 7f003136f6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f002c115440 0x7f002c11d770 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:22.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.604+0000 7f003136f6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f002c115440 0x7f002c11d770 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:46812/0 (socket says 10.20.193.27:46812) 2026-02-13T02:23:22.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.604+0000 7f003136f6c0 1 -- 10.20.193.27:0/1673874255 learned_addr learned my addr 10.20.193.27:0/1673874255 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:23:22.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.604+0000 7f0031b706c0 1 --2- 10.20.193.27:0/1673874255 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f002c10f230 0x7f002c11d230 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:22.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.604+0000 7f003136f6c0 1 -- 10.20.193.27:0/1673874255 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f002c122b30 msgr2=0x7f002c122fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:22.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.604+0000 7f00323716c0 1 --2- 10.20.193.27:0/1673874255 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f002c122b30 0x7f002c122fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:22.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.604+0000 7f003136f6c0 1 --2- 10.20.193.27:0/1673874255 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f002c122b30 0x7f002c122fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:22.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.604+0000 7f003136f6c0 1 -- 10.20.193.27:0/1673874255 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f002c10f230 msgr2=0x7f002c11d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:22.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.604+0000 7f003136f6c0 1 --2- 10.20.193.27:0/1673874255 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f002c10f230 0x7f002c11d230 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:22.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.604+0000 7f003136f6c0 1 -- 10.20.193.27:0/1673874255 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f002800a660 con 0x7f002c115440 2026-02-13T02:23:22.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.604+0000 7f00323716c0 1 --2- 10.20.193.27:0/1673874255 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f002c122b30 0x7f002c122fd0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:23:22.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.604+0000 7f003136f6c0 1 --2- 10.20.193.27:0/1673874255 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f002c115440 0x7f002c11d770 secure :-1 s=READY pgs=365 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0020005dc0 tx=0x7f002000bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:22.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.604+0000 7f001affd6c0 1 -- 10.20.193.27:0/1673874255 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0020004250 con 0x7f002c115440 2026-02-13T02:23:22.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.604+0000 7f001affd6c0 1 -- 10.20.193.27:0/1673874255 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f0020009070 con 0x7f002c115440 2026-02-13T02:23:22.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.604+0000 7f0033dff6c0 1 -- 10.20.193.27:0/1673874255 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f002c163d20 con 0x7f002c115440 2026-02-13T02:23:22.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.604+0000 7f001affd6c0 1 -- 10.20.193.27:0/1673874255 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f002000cd50 con 0x7f002c115440 2026-02-13T02:23:22.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.604+0000 7f0033dff6c0 1 -- 10.20.193.27:0/1673874255 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f002c1641b0 con 0x7f002c115440 2026-02-13T02:23:22.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.605+0000 7f0033dff6c0 1 -- 10.20.193.27:0/1673874255 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f002c110500 con 0x7f002c115440 2026-02-13T02:23:22.612 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.606+0000 7f001affd6c0 1 -- 10.20.193.27:0/1673874255 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f00200028c0 con 0x7f002c115440 2026-02-13T02:23:22.612 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.607+0000 7f001affd6c0 1 --2- 10.20.193.27:0/1673874255 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f00080b5690 0x7f00080d5b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:22.612 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.607+0000 7f0031b706c0 1 --2- 10.20.193.27:0/1673874255 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f00080b5690 0x7f00080d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:22.612 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.607+0000 7f001affd6c0 1 -- 10.20.193.27:0/1673874255 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f00200e3cb0 con 0x7f002c115440 2026-02-13T02:23:22.613 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.607+0000 7f0031b706c0 1 --2- 10.20.193.27:0/1673874255 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f00080b5690 0x7f00080d5b40 secure :-1 s=READY pgs=246 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f001c005980 tx=0x7f001c0094b0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:22.613 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.608+0000 7f001affd6c0 1 -- 10.20.193.27:0/1673874255 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f00200a65d0 con 0x7f002c115440 2026-02-13T02:23:22.725 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.723+0000 7f0033dff6c0 1 -- 10.20.193.27:0/1673874255 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f002c10f600 con 0x7f002c115440 2026-02-13T02:23:22.725 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.723+0000 7f001affd6c0 1 -- 10.20.193.27:0/1673874255 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f00200a9780 con 0x7f002c115440 2026-02-13T02:23:22.726 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:23:22.726 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:23:22.727 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.724+0000 7f0033dff6c0 1 -- 10.20.193.27:0/1673874255 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f00080b5690 msgr2=0x7f00080d5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:22.727 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.724+0000 7f0033dff6c0 1 --2- 10.20.193.27:0/1673874255 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f00080b5690 0x7f00080d5b40 secure :-1 s=READY pgs=246 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f001c005980 tx=0x7f001c0094b0 comp rx=0 tx=0).stop 2026-02-13T02:23:22.727 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.724+0000 7f0033dff6c0 1 -- 10.20.193.27:0/1673874255 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f002c115440 msgr2=0x7f002c11d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:22.727 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.724+0000 7f0033dff6c0 1 --2- 10.20.193.27:0/1673874255 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f002c115440 0x7f002c11d770 secure :-1 s=READY pgs=365 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0020005dc0 tx=0x7f002000bbb0 comp rx=0 tx=0).stop 2026-02-13T02:23:22.727 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.724+0000 7f0033dff6c0 1 -- 10.20.193.27:0/1673874255 shutdown_connections 2026-02-13T02:23:22.727 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.724+0000 7f0033dff6c0 1 --2- 10.20.193.27:0/1673874255 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f00080b5690 0x7f00080d5b40 unknown :-1 s=CLOSED pgs=246 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:22.727 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.724+0000 7f0033dff6c0 1 --2- 10.20.193.27:0/1673874255 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f002c122b30 0x7f002c122fd0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:22.727 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.724+0000 7f0033dff6c0 1 --2- 10.20.193.27:0/1673874255 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f002c115440 0x7f002c11d770 unknown :-1 s=CLOSED pgs=365 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:22.728 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.724+0000 7f0033dff6c0 1 --2- 10.20.193.27:0/1673874255 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f002c10f230 0x7f002c11d230 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:22.728 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.724+0000 7f0033dff6c0 1 -- 10.20.193.27:0/1673874255 >> 10.20.193.27:0/1673874255 conn(0x7f002c0ecb30 msgr2=0x7f002c10cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:22.728 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.724+0000 7f0033dff6c0 1 -- 10.20.193.27:0/1673874255 shutdown_connections 2026-02-13T02:23:22.728 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:22.724+0000 7f0033dff6c0 1 -- 10.20.193.27:0/1673874255 wait complete. 2026-02-13T02:23:23.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:23 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1673874255' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:23.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:23 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1673874255' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:23.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:23 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1673874255' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:24.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:24 trial028 ceph-mon[26701]: pgmap v220: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 2 op/s 2026-02-13T02:23:24.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:24 trial107 ceph-mon[27422]: pgmap v220: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 2 op/s 2026-02-13T02:23:24.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:24 trial027 ceph-mon[24567]: pgmap v220: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 2 op/s 2026-02-13T02:23:25.767 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:23:25.887 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:23:26.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.008+0000 7f50151686c0 1 -- 10.20.193.27:0/693224113 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5010115420 msgr2=0x7f5010175dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:26.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.008+0000 7f50151686c0 1 --2- 10.20.193.27:0/693224113 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5010115420 0x7f5010175dc0 secure :-1 s=READY pgs=366 gs=1 cs=0 l=1 c_cookie=17208161051841148325 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f500000aa70 tx=0x7f500002fde0 comp rx=0 tx=0).stop 2026-02-13T02:23:26.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.008+0000 7f50151686c0 1 -- 10.20.193.27:0/693224113 shutdown_connections 2026-02-13T02:23:26.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.008+0000 7f50151686c0 1 --2- 10.20.193.27:0/693224113 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5010115420 0x7f5010175dc0 unknown :-1 s=CLOSED pgs=366 gs=1 cs=0 l=1 c_cookie=17208161051841148325 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:26.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.008+0000 7f50151686c0 1 --2- 10.20.193.27:0/693224113 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f501010fc00 0x7f5010114c40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:26.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.008+0000 7f50151686c0 1 --2- 10.20.193.27:0/693224113 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f501010f210 0x7f501010f5e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:26.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.008+0000 7f50151686c0 1 -- 10.20.193.27:0/693224113 >> 10.20.193.27:0/693224113 conn(0x7f50100ecb30 msgr2=0x7f501010cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:26.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.008+0000 7f50151686c0 1 -- 10.20.193.27:0/693224113 shutdown_connections 2026-02-13T02:23:26.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.008+0000 7f50151686c0 1 -- 10.20.193.27:0/693224113 wait complete. 2026-02-13T02:23:26.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.009+0000 7f50151686c0 1 Processor -- start 2026-02-13T02:23:26.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.009+0000 7f50151686c0 1 -- start start 2026-02-13T02:23:26.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.009+0000 7f50151686c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f501010f210 0x7f501011d260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:26.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.009+0000 7f50151686c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5010115420 0x7f501011d7a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:26.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.009+0000 7f50151686c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5010122b60 0x7f5010123000 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:26.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.009+0000 7f50151686c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5010123570 con 0x7f5010115420 2026-02-13T02:23:26.014 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.009+0000 7f50151686c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f50101236e0 con 0x7f501010f210 2026-02-13T02:23:26.014 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.009+0000 7f50151686c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f50101238a0 con 0x7f5010122b60 2026-02-13T02:23:26.014 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.009+0000 7f500e5756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5010115420 0x7f501011d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:26.014 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.009+0000 7f500e5756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5010115420 0x7f501011d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:46824/0 (socket says 10.20.193.27:46824) 2026-02-13T02:23:26.014 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.009+0000 7f500e5756c0 1 -- 10.20.193.27:0/3604028644 learned_addr learned my addr 10.20.193.27:0/3604028644 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:23:26.014 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.009+0000 7f500ed766c0 1 --2- 10.20.193.27:0/3604028644 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f501010f210 0x7f501011d260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:26.014 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.009+0000 7f500f5776c0 1 --2- 10.20.193.27:0/3604028644 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5010122b60 0x7f5010123000 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:26.014 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.009+0000 7f500e5756c0 1 -- 10.20.193.27:0/3604028644 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5010122b60 msgr2=0x7f5010123000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:26.014 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.009+0000 7f500e5756c0 1 --2- 10.20.193.27:0/3604028644 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5010122b60 0x7f5010123000 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:26.014 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.009+0000 7f500e5756c0 1 -- 10.20.193.27:0/3604028644 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f501010f210 msgr2=0x7f501011d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:26.014 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.009+0000 7f500e5756c0 1 --2- 10.20.193.27:0/3604028644 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f501010f210 0x7f501011d260 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:26.014 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.009+0000 7f500e5756c0 1 -- 10.20.193.27:0/3604028644 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f500000a660 con 0x7f5010115420 2026-02-13T02:23:26.015 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.009+0000 7f500f5776c0 1 --2- 10.20.193.27:0/3604028644 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5010122b60 0x7f5010123000 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:23:26.015 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.009+0000 7f500ed766c0 1 --2- 10.20.193.27:0/3604028644 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f501010f210 0x7f501011d260 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:23:26.015 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.009+0000 7f500e5756c0 1 --2- 10.20.193.27:0/3604028644 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5010115420 0x7f501011d7a0 secure :-1 s=READY pgs=367 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4ff800b750 tx=0x7f4ff800bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:26.015 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.009+0000 7f4fe7fff6c0 1 -- 10.20.193.27:0/3604028644 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4ff8004a80 con 0x7f5010115420 2026-02-13T02:23:26.015 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.009+0000 7f4fe7fff6c0 1 -- 10.20.193.27:0/3604028644 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4ff8009030 con 0x7f5010115420 2026-02-13T02:23:26.015 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.009+0000 7f50151686c0 1 -- 10.20.193.27:0/3604028644 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5010163d50 con 0x7f5010115420 2026-02-13T02:23:26.015 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.009+0000 7f4fe7fff6c0 1 -- 10.20.193.27:0/3604028644 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4ff800cd00 con 0x7f5010115420 2026-02-13T02:23:26.015 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.009+0000 7f50151686c0 1 -- 10.20.193.27:0/3604028644 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50101641e0 con 0x7f5010115420 2026-02-13T02:23:26.015 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.010+0000 7f50151686c0 1 -- 10.20.193.27:0/3604028644 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50101104e0 con 0x7f5010115420 2026-02-13T02:23:26.016 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.011+0000 7f4fe7fff6c0 1 -- 10.20.193.27:0/3604028644 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 12) ==== 157117+0+0 (secure 0 0 0) 0x7f4ff8005020 con 0x7f5010115420 2026-02-13T02:23:26.016 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.012+0000 7f4fe7fff6c0 1 --2- 10.20.193.27:0/3604028644 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f4fe00b5710 0x7f4fe00d5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:26.016 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.012+0000 7f4fe7fff6c0 1 -- 10.20.193.27:0/3604028644 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f4ff80e3bf0 con 0x7f5010115420 2026-02-13T02:23:26.016 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.012+0000 7f500ed766c0 1 --2- 10.20.193.27:0/3604028644 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f4fe00b5710 0x7f4fe00d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:26.017 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.012+0000 7f4fe7fff6c0 1 -- 10.20.193.27:0/3604028644 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f4ff80a96c0 con 0x7f5010115420 2026-02-13T02:23:26.017 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.012+0000 7f500ed766c0 1 --2- 10.20.193.27:0/3604028644 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f4fe00b5710 0x7f4fe00d5bc0 secure :-1 s=READY pgs=247 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5004004750 tx=0x7f5004009260 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:26.127 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.126+0000 7f50151686c0 1 -- 10.20.193.27:0/3604028644 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f501011e3b0 con 0x7f5010115420 2026-02-13T02:23:26.128 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.126+0000 7f4fe7fff6c0 1 -- 10.20.193.27:0/3604028644 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f4ff80a98a0 con 0x7f5010115420 2026-02-13T02:23:26.128 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:23:26.128 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:23:26.130 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.127+0000 7f50151686c0 1 -- 10.20.193.27:0/3604028644 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f4fe00b5710 msgr2=0x7f4fe00d5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:26.130 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.127+0000 7f50151686c0 1 --2- 10.20.193.27:0/3604028644 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f4fe00b5710 0x7f4fe00d5bc0 secure :-1 s=READY pgs=247 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5004004750 tx=0x7f5004009260 comp rx=0 tx=0).stop 2026-02-13T02:23:26.130 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.127+0000 7f50151686c0 1 -- 10.20.193.27:0/3604028644 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5010115420 msgr2=0x7f501011d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:26.130 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.127+0000 7f50151686c0 1 --2- 10.20.193.27:0/3604028644 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5010115420 0x7f501011d7a0 secure :-1 s=READY pgs=367 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4ff800b750 tx=0x7f4ff800bc20 comp rx=0 tx=0).stop 2026-02-13T02:23:26.130 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.127+0000 7f50151686c0 1 -- 10.20.193.27:0/3604028644 shutdown_connections 2026-02-13T02:23:26.131 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.127+0000 7f50151686c0 1 --2- 10.20.193.27:0/3604028644 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f4fe00b5710 0x7f4fe00d5bc0 unknown :-1 s=CLOSED pgs=247 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:26.131 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.127+0000 7f50151686c0 1 --2- 10.20.193.27:0/3604028644 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5010122b60 0x7f5010123000 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:26.131 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.127+0000 7f50151686c0 1 --2- 10.20.193.27:0/3604028644 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5010115420 0x7f501011d7a0 unknown :-1 s=CLOSED pgs=367 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:26.131 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.127+0000 7f50151686c0 1 --2- 10.20.193.27:0/3604028644 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f501010f210 0x7f501011d260 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:26.131 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.127+0000 7f50151686c0 1 -- 10.20.193.27:0/3604028644 >> 10.20.193.27:0/3604028644 conn(0x7f50100ecb30 msgr2=0x7f501010cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:26.131 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.127+0000 7f50151686c0 1 -- 10.20.193.27:0/3604028644 shutdown_connections 2026-02-13T02:23:26.131 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:26.127+0000 7f50151686c0 1 -- 10.20.193.27:0/3604028644 wait complete. 2026-02-13T02:23:26.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:26 trial027 ceph-mon[24567]: pgmap v221: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 2 op/s 2026-02-13T02:23:26.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:26 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3604028644' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:26.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:26 trial028 ceph-mon[26701]: pgmap v221: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 2 op/s 2026-02-13T02:23:26.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:26 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3604028644' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:26.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:26 trial107 ceph-mon[27422]: pgmap v221: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 2 op/s 2026-02-13T02:23:26.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:26 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3604028644' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:28.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:28 trial028 ceph-mon[26701]: pgmap v222: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.4 KiB/s rd, 193 B/s wr, 3 op/s 2026-02-13T02:23:28.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:28 trial107 ceph-mon[27422]: pgmap v222: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.4 KiB/s rd, 193 B/s wr, 3 op/s 2026-02-13T02:23:28.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:28 trial027 ceph-mon[24567]: pgmap v222: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.4 KiB/s rd, 193 B/s wr, 3 op/s 2026-02-13T02:23:29.170 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:23:29.292 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:23:29.409 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.405+0000 7f530e5406c0 1 -- 10.20.193.27:0/1067358998 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f530810f230 msgr2=0x7f530810f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:29.410 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.405+0000 7f530e5406c0 1 --2- 10.20.193.27:0/1067358998 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f530810f230 0x7f530810f600 secure :-1 s=READY pgs=368 gs=1 cs=0 l=1 c_cookie=4404447742866804381 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f52f40099a0 tx=0x7f52f402ed20 comp rx=0 tx=0).stop 2026-02-13T02:23:29.410 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.406+0000 7f530e5406c0 1 -- 10.20.193.27:0/1067358998 shutdown_connections 2026-02-13T02:23:29.410 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.406+0000 7f530e5406c0 1 --2- 10.20.193.27:0/1067358998 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5308115440 0x7f5308175de0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:29.410 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.406+0000 7f530e5406c0 1 --2- 10.20.193.27:0/1067358998 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f530810fc20 0x7f5308114c60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:29.410 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.406+0000 7f530e5406c0 1 --2- 10.20.193.27:0/1067358998 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f530810f230 0x7f530810f600 unknown :-1 s=CLOSED pgs=368 gs=1 cs=0 l=1 c_cookie=4404447742866804381 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:29.410 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.406+0000 7f530e5406c0 1 -- 10.20.193.27:0/1067358998 >> 10.20.193.27:0/1067358998 conn(0x7f53080ecb30 msgr2=0x7f530810cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:29.410 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.406+0000 7f530e5406c0 1 -- 10.20.193.27:0/1067358998 shutdown_connections 2026-02-13T02:23:29.410 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.406+0000 7f530e5406c0 1 -- 10.20.193.27:0/1067358998 wait complete. 2026-02-13T02:23:29.410 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.406+0000 7f530e5406c0 1 Processor -- start 2026-02-13T02:23:29.410 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.406+0000 7f530e5406c0 1 -- start start 2026-02-13T02:23:29.411 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.406+0000 7f530e5406c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f530810f230 0x7f530811d320 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:29.411 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.406+0000 7f530e5406c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5308115440 0x7f530811d860 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:29.411 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.406+0000 7f530e5406c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5308122c20 0x7f53081230c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:29.411 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.406+0000 7f530e5406c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5308123630 con 0x7f5308122c20 2026-02-13T02:23:29.411 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.406+0000 7f530e5406c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f53081237a0 con 0x7f5308115440 2026-02-13T02:23:29.411 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.406+0000 7f530e5406c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5308123960 con 0x7f530810f230 2026-02-13T02:23:29.411 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.406+0000 7f530cab66c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5308122c20 0x7f53081230c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:29.411 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.406+0000 7f530cab66c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5308122c20 0x7f53081230c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:35566/0 (socket says 10.20.193.27:35566) 2026-02-13T02:23:29.411 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.406+0000 7f530cab66c0 1 -- 10.20.193.27:0/151069435 learned_addr learned my addr 10.20.193.27:0/151069435 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:23:29.411 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.406+0000 7f53077fe6c0 1 --2- 10.20.193.27:0/151069435 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5308115440 0x7f530811d860 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:29.411 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.406+0000 7f5307fff6c0 1 --2- 10.20.193.27:0/151069435 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f530810f230 0x7f530811d320 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:29.411 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.406+0000 7f530cab66c0 1 -- 10.20.193.27:0/151069435 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f530810f230 msgr2=0x7f530811d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:29.411 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.406+0000 7f530cab66c0 1 --2- 10.20.193.27:0/151069435 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f530810f230 0x7f530811d320 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:29.412 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.406+0000 7f530cab66c0 1 -- 10.20.193.27:0/151069435 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5308115440 msgr2=0x7f530811d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:29.412 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.406+0000 7f530cab66c0 1 --2- 10.20.193.27:0/151069435 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5308115440 0x7f530811d860 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:29.412 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.406+0000 7f530cab66c0 1 -- 10.20.193.27:0/151069435 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52f4009590 con 0x7f5308122c20 2026-02-13T02:23:29.412 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.406+0000 7f530cab66c0 1 --2- 10.20.193.27:0/151069435 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5308122c20 0x7f53081230c0 secure :-1 s=READY pgs=369 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f52fc006ea0 tx=0x7f52fc00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:29.412 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.406+0000 7f53057fa6c0 1 -- 10.20.193.27:0/151069435 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f52fc003a30 con 0x7f5308122c20 2026-02-13T02:23:29.412 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.406+0000 7f53057fa6c0 1 -- 10.20.193.27:0/151069435 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f52fc0054f0 con 0x7f5308122c20 2026-02-13T02:23:29.412 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.406+0000 7f530e5406c0 1 -- 10.20.193.27:0/151069435 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5308163e10 con 0x7f5308122c20 2026-02-13T02:23:29.412 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.407+0000 7f53057fa6c0 1 -- 10.20.193.27:0/151069435 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f52fc00dcd0 con 0x7f5308122c20 2026-02-13T02:23:29.412 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.407+0000 7f530e5406c0 1 -- 10.20.193.27:0/151069435 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53081642a0 con 0x7f5308122c20 2026-02-13T02:23:29.412 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.407+0000 7f530e5406c0 1 -- 10.20.193.27:0/151069435 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5308110500 con 0x7f5308122c20 2026-02-13T02:23:29.412 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.408+0000 7f53057fa6c0 1 -- 10.20.193.27:0/151069435 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f52fc005030 con 0x7f5308122c20 2026-02-13T02:23:29.413 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.409+0000 7f53057fa6c0 1 --2- 10.20.193.27:0/151069435 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f52dc0b5600 0x7f52dc0d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:29.413 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.409+0000 7f5307fff6c0 1 --2- 10.20.193.27:0/151069435 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f52dc0b5600 0x7f52dc0d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:29.413 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.409+0000 7f53057fa6c0 1 -- 10.20.193.27:0/151069435 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f52fc0e4b10 con 0x7f5308122c20 2026-02-13T02:23:29.414 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.409+0000 7f53057fa6c0 1 -- 10.20.193.27:0/151069435 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f52fc0aa780 con 0x7f5308122c20 2026-02-13T02:23:29.414 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.409+0000 7f5307fff6c0 1 --2- 10.20.193.27:0/151069435 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f52dc0b5600 0x7f52dc0d5ab0 secure :-1 s=READY pgs=248 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f52f40092e0 tx=0x7f52f4005c00 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:29.525 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.523+0000 7f530e5406c0 1 -- 10.20.193.27:0/151069435 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f530810f600 con 0x7f5308122c20 2026-02-13T02:23:29.525 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.523+0000 7f53057fa6c0 1 -- 10.20.193.27:0/151069435 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f52fc0aa960 con 0x7f5308122c20 2026-02-13T02:23:29.526 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:23:29.526 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:23:29.527 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.525+0000 7f530e5406c0 1 -- 10.20.193.27:0/151069435 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f52dc0b5600 msgr2=0x7f52dc0d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:29.528 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.525+0000 7f530e5406c0 1 --2- 10.20.193.27:0/151069435 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f52dc0b5600 0x7f52dc0d5ab0 secure :-1 s=READY pgs=248 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f52f40092e0 tx=0x7f52f4005c00 comp rx=0 tx=0).stop 2026-02-13T02:23:29.528 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.525+0000 7f530e5406c0 1 -- 10.20.193.27:0/151069435 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5308122c20 msgr2=0x7f53081230c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:29.528 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.525+0000 7f530e5406c0 1 --2- 10.20.193.27:0/151069435 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5308122c20 0x7f53081230c0 secure :-1 s=READY pgs=369 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f52fc006ea0 tx=0x7f52fc00cca0 comp rx=0 tx=0).stop 2026-02-13T02:23:29.528 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.525+0000 7f530e5406c0 1 -- 10.20.193.27:0/151069435 shutdown_connections 2026-02-13T02:23:29.528 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.525+0000 7f530e5406c0 1 --2- 10.20.193.27:0/151069435 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f52dc0b5600 0x7f52dc0d5ab0 unknown :-1 s=CLOSED pgs=248 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:29.528 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.525+0000 7f530e5406c0 1 --2- 10.20.193.27:0/151069435 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5308122c20 0x7f53081230c0 unknown :-1 s=CLOSED pgs=369 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:29.528 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.525+0000 7f530e5406c0 1 --2- 10.20.193.27:0/151069435 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5308115440 0x7f530811d860 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:29.528 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.525+0000 7f530e5406c0 1 --2- 10.20.193.27:0/151069435 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f530810f230 0x7f530811d320 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:29.528 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.525+0000 7f530e5406c0 1 -- 10.20.193.27:0/151069435 >> 10.20.193.27:0/151069435 conn(0x7f53080ecb30 msgr2=0x7f530810cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:29.529 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.525+0000 7f530e5406c0 1 -- 10.20.193.27:0/151069435 shutdown_connections 2026-02-13T02:23:29.529 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:29.525+0000 7f530e5406c0 1 -- 10.20.193.27:0/151069435 wait complete. 2026-02-13T02:23:30.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:30 trial028 ceph-mon[26701]: pgmap v223: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2026-02-13T02:23:30.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:30 trial028 ceph-mon[26701]: mgrmap e13: x(active, since 3m), standbys: z, y 2026-02-13T02:23:30.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:30 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/151069435' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:30.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:30 trial107 ceph-mon[27422]: pgmap v223: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2026-02-13T02:23:30.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:30 trial107 ceph-mon[27422]: mgrmap e13: x(active, since 3m), standbys: z, y 2026-02-13T02:23:30.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:30 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/151069435' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:30.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:30 trial027 ceph-mon[24567]: pgmap v223: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2026-02-13T02:23:30.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:30 trial027 ceph-mon[24567]: mgrmap e13: x(active, since 3m), standbys: z, y 2026-02-13T02:23:30.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:30 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/151069435' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:32.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:32 trial028 ceph-mon[26701]: pgmap v224: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2026-02-13T02:23:32.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:32 trial107 ceph-mon[27422]: pgmap v224: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2026-02-13T02:23:32.568 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:23:32.593 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:32 trial027 ceph-mon[24567]: pgmap v224: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2026-02-13T02:23:32.689 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:23:32.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.802+0000 7f16d6f276c0 1 -- 10.20.193.27:0/1251404524 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f16d010f230 msgr2=0x7f16d010f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:32.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.802+0000 7f16d6f276c0 1 --2- 10.20.193.27:0/1251404524 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f16d010f230 0x7f16d010f600 secure :-1 s=READY pgs=370 gs=1 cs=0 l=1 c_cookie=11695679852594660166 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f16c00099a0 tx=0x7f16c002ed20 comp rx=0 tx=0).stop 2026-02-13T02:23:32.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.803+0000 7f16d6f276c0 1 -- 10.20.193.27:0/1251404524 shutdown_connections 2026-02-13T02:23:32.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.803+0000 7f16d6f276c0 1 --2- 10.20.193.27:0/1251404524 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f16d0115440 0x7f16d0175de0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:32.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.803+0000 7f16d6f276c0 1 --2- 10.20.193.27:0/1251404524 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f16d010fc20 0x7f16d0114c60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:32.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.803+0000 7f16d6f276c0 1 --2- 10.20.193.27:0/1251404524 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f16d010f230 0x7f16d010f600 unknown :-1 s=CLOSED pgs=370 gs=1 cs=0 l=1 c_cookie=11695679852594660166 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:32.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.803+0000 7f16d6f276c0 1 -- 10.20.193.27:0/1251404524 >> 10.20.193.27:0/1251404524 conn(0x7f16d00ecb30 msgr2=0x7f16d010cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:32.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.803+0000 7f16d6f276c0 1 -- 10.20.193.27:0/1251404524 shutdown_connections 2026-02-13T02:23:32.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.803+0000 7f16d6f276c0 1 -- 10.20.193.27:0/1251404524 wait complete. 2026-02-13T02:23:32.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.803+0000 7f16d6f276c0 1 Processor -- start 2026-02-13T02:23:32.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.803+0000 7f16d6f276c0 1 -- start start 2026-02-13T02:23:32.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.803+0000 7f16d6f276c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f16d010f230 0x7f16d011d230 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:32.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.803+0000 7f16d6f276c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f16d0115440 0x7f16d011d770 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:32.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.803+0000 7f16d6f276c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f16d0122b30 0x7f16d0122fd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:32.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.803+0000 7f16d6f276c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f16d0123540 con 0x7f16d0115440 2026-02-13T02:23:32.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.803+0000 7f16d6f276c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f16d01236b0 con 0x7f16d0122b30 2026-02-13T02:23:32.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.803+0000 7f16d6f276c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f16d0123870 con 0x7f16d010f230 2026-02-13T02:23:32.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.803+0000 7f16c7fff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f16d0115440 0x7f16d011d770 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:32.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.803+0000 7f16c7fff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f16d0115440 0x7f16d011d770 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:35580/0 (socket says 10.20.193.27:35580) 2026-02-13T02:23:32.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.803+0000 7f16c7fff6c0 1 -- 10.20.193.27:0/42327462 learned_addr learned my addr 10.20.193.27:0/42327462 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:23:32.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.803+0000 7f16d4c986c0 1 --2- 10.20.193.27:0/42327462 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f16d010f230 0x7f16d011d230 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:32.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.803+0000 7f16c7fff6c0 1 -- 10.20.193.27:0/42327462 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f16d010f230 msgr2=0x7f16d011d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:32.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.803+0000 7f16c7fff6c0 1 --2- 10.20.193.27:0/42327462 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f16d010f230 0x7f16d011d230 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:32.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.803+0000 7f16c7fff6c0 1 -- 10.20.193.27:0/42327462 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f16d0122b30 msgr2=0x7f16d0122fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:32.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.803+0000 7f16c7fff6c0 1 --2- 10.20.193.27:0/42327462 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f16d0122b30 0x7f16d0122fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:32.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.803+0000 7f16c7fff6c0 1 -- 10.20.193.27:0/42327462 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c0009590 con 0x7f16d0115440 2026-02-13T02:23:32.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.803+0000 7f16c7fff6c0 1 --2- 10.20.193.27:0/42327462 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f16d0115440 0x7f16d011d770 secure :-1 s=READY pgs=371 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f16b8005dc0 tx=0x7f16b800bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:32.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.803+0000 7f16c5ffb6c0 1 -- 10.20.193.27:0/42327462 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f16b8004250 con 0x7f16d0115440 2026-02-13T02:23:32.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.803+0000 7f16c5ffb6c0 1 -- 10.20.193.27:0/42327462 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f16b8009070 con 0x7f16d0115440 2026-02-13T02:23:32.809 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.803+0000 7f16d6f276c0 1 -- 10.20.193.27:0/42327462 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d0163d20 con 0x7f16d0115440 2026-02-13T02:23:32.809 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.803+0000 7f16c5ffb6c0 1 -- 10.20.193.27:0/42327462 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f16b800cd50 con 0x7f16d0115440 2026-02-13T02:23:32.809 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.804+0000 7f16d6f276c0 1 -- 10.20.193.27:0/42327462 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d0164200 con 0x7f16d0115440 2026-02-13T02:23:32.809 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.804+0000 7f16d6f276c0 1 -- 10.20.193.27:0/42327462 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16d0110500 con 0x7f16d0115440 2026-02-13T02:23:32.810 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.805+0000 7f16c5ffb6c0 1 -- 10.20.193.27:0/42327462 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f16b80028c0 con 0x7f16d0115440 2026-02-13T02:23:32.810 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.806+0000 7f16c5ffb6c0 1 --2- 10.20.193.27:0/42327462 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f16ac0b5680 0x7f16ac0d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:32.810 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.806+0000 7f16d4c986c0 1 --2- 10.20.193.27:0/42327462 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f16ac0b5680 0x7f16ac0d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:32.810 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.806+0000 7f16c5ffb6c0 1 -- 10.20.193.27:0/42327462 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f16b80e3a90 con 0x7f16d0115440 2026-02-13T02:23:32.810 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.806+0000 7f16d4c986c0 1 --2- 10.20.193.27:0/42327462 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f16ac0b5680 0x7f16ac0d5b30 secure :-1 s=READY pgs=249 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f16c0006010 tx=0x7f16c0005de0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:32.810 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.806+0000 7f16c5ffb6c0 1 -- 10.20.193.27:0/42327462 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f16b80a96c0 con 0x7f16d0115440 2026-02-13T02:23:32.922 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.921+0000 7f16d6f276c0 1 -- 10.20.193.27:0/42327462 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f16d010f600 con 0x7f16d0115440 2026-02-13T02:23:32.922 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.921+0000 7f16c5ffb6c0 1 -- 10.20.193.27:0/42327462 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f16b80a98a0 con 0x7f16d0115440 2026-02-13T02:23:32.923 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:23:32.923 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:23:32.924 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.922+0000 7f16d6f276c0 1 -- 10.20.193.27:0/42327462 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f16ac0b5680 msgr2=0x7f16ac0d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:32.924 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.922+0000 7f16d6f276c0 1 --2- 10.20.193.27:0/42327462 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f16ac0b5680 0x7f16ac0d5b30 secure :-1 s=READY pgs=249 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f16c0006010 tx=0x7f16c0005de0 comp rx=0 tx=0).stop 2026-02-13T02:23:32.924 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.922+0000 7f16d6f276c0 1 -- 10.20.193.27:0/42327462 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f16d0115440 msgr2=0x7f16d011d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:32.924 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.922+0000 7f16d6f276c0 1 --2- 10.20.193.27:0/42327462 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f16d0115440 0x7f16d011d770 secure :-1 s=READY pgs=371 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f16b8005dc0 tx=0x7f16b800bbb0 comp rx=0 tx=0).stop 2026-02-13T02:23:32.924 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.922+0000 7f16d6f276c0 1 -- 10.20.193.27:0/42327462 shutdown_connections 2026-02-13T02:23:32.924 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.922+0000 7f16d6f276c0 1 --2- 10.20.193.27:0/42327462 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f16ac0b5680 0x7f16ac0d5b30 unknown :-1 s=CLOSED pgs=249 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:32.924 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.922+0000 7f16d6f276c0 1 --2- 10.20.193.27:0/42327462 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f16d0122b30 0x7f16d0122fd0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:32.924 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.922+0000 7f16d6f276c0 1 --2- 10.20.193.27:0/42327462 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f16d0115440 0x7f16d011d770 unknown :-1 s=CLOSED pgs=371 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:32.925 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.922+0000 7f16d6f276c0 1 --2- 10.20.193.27:0/42327462 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f16d010f230 0x7f16d011d230 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:32.925 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.922+0000 7f16d6f276c0 1 -- 10.20.193.27:0/42327462 >> 10.20.193.27:0/42327462 conn(0x7f16d00ecb30 msgr2=0x7f16d010cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:32.925 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.922+0000 7f16d6f276c0 1 -- 10.20.193.27:0/42327462 shutdown_connections 2026-02-13T02:23:32.925 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:32.922+0000 7f16d6f276c0 1 -- 10.20.193.27:0/42327462 wait complete. 2026-02-13T02:23:33.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:33 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/42327462' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:33.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:33 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/42327462' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:33.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:33 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/42327462' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:34.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:34 trial028 ceph-mon[26701]: pgmap v225: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2026-02-13T02:23:34.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:34 trial107 ceph-mon[27422]: pgmap v225: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2026-02-13T02:23:34.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:34 trial027 ceph-mon[24567]: pgmap v225: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2026-02-13T02:23:35.966 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:23:36.086 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:23:36.203 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.199+0000 7f12e89556c0 1 -- 10.20.193.27:0/2376403414 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f12e010f200 msgr2=0x7f12e010f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:36.203 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.199+0000 7f12e89556c0 1 --2- 10.20.193.27:0/2376403414 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f12e010f200 0x7f12e010f5d0 secure :-1 s=READY pgs=372 gs=1 cs=0 l=1 c_cookie=4794179908996940700 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f12d80099a0 tx=0x7f12d802ed20 comp rx=0 tx=0).stop 2026-02-13T02:23:36.203 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.199+0000 7f12e89556c0 1 -- 10.20.193.27:0/2376403414 shutdown_connections 2026-02-13T02:23:36.203 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.199+0000 7f12e89556c0 1 --2- 10.20.193.27:0/2376403414 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f12e0115410 0x7f12e0175db0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:36.203 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.199+0000 7f12e89556c0 1 --2- 10.20.193.27:0/2376403414 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f12e010fbf0 0x7f12e0114c30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:36.203 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.199+0000 7f12e89556c0 1 --2- 10.20.193.27:0/2376403414 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f12e010f200 0x7f12e010f5d0 unknown :-1 s=CLOSED pgs=372 gs=1 cs=0 l=1 c_cookie=4794179908996940700 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:36.204 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.199+0000 7f12e89556c0 1 -- 10.20.193.27:0/2376403414 >> 10.20.193.27:0/2376403414 conn(0x7f12e00ecb00 msgr2=0x7f12e010cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:36.204 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.199+0000 7f12e89556c0 1 -- 10.20.193.27:0/2376403414 shutdown_connections 2026-02-13T02:23:36.204 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.199+0000 7f12e89556c0 1 -- 10.20.193.27:0/2376403414 wait complete. 2026-02-13T02:23:36.204 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.200+0000 7f12e89556c0 1 Processor -- start 2026-02-13T02:23:36.204 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.200+0000 7f12e89556c0 1 -- start start 2026-02-13T02:23:36.204 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.200+0000 7f12e89556c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f12e010f200 0x7f12e011d390 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:36.204 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.200+0000 7f12e89556c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f12e0115410 0x7f12e011d8d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:36.204 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.200+0000 7f12e89556c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f12e0122c90 0x7f12e0123130 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:36.204 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.200+0000 7f12e89556c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f12e01236a0 con 0x7f12e010f200 2026-02-13T02:23:36.204 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.200+0000 7f12e89556c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f12e0123810 con 0x7f12e0115410 2026-02-13T02:23:36.204 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.200+0000 7f12e89556c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f12e01239d0 con 0x7f12e0122c90 2026-02-13T02:23:36.205 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.200+0000 7f12e66c66c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f12e010f200 0x7f12e011d390 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:36.205 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.200+0000 7f12e66c66c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f12e010f200 0x7f12e011d390 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:35602/0 (socket says 10.20.193.27:35602) 2026-02-13T02:23:36.205 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.200+0000 7f12e66c66c0 1 -- 10.20.193.27:0/2124265908 learned_addr learned my addr 10.20.193.27:0/2124265908 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:23:36.205 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.200+0000 7f12e66c66c0 1 -- 10.20.193.27:0/2124265908 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f12e0122c90 msgr2=0x7f12e0123130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:36.205 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.200+0000 7f12e66c66c0 1 --2- 10.20.193.27:0/2124265908 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f12e0122c90 0x7f12e0123130 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:36.205 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.200+0000 7f12e66c66c0 1 -- 10.20.193.27:0/2124265908 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f12e0115410 msgr2=0x7f12e011d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:36.205 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.200+0000 7f12e66c66c0 1 --2- 10.20.193.27:0/2124265908 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f12e0115410 0x7f12e011d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:36.205 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.200+0000 7f12e66c66c0 1 -- 10.20.193.27:0/2124265908 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12d8009590 con 0x7f12e010f200 2026-02-13T02:23:36.205 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.200+0000 7f12e66c66c0 1 --2- 10.20.193.27:0/2124265908 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f12e010f200 0x7f12e011d390 secure :-1 s=READY pgs=373 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f12d802f230 tx=0x7f12d802fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:36.205 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.200+0000 7f12cb7fe6c0 1 -- 10.20.193.27:0/2124265908 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f12d8002ba0 con 0x7f12e010f200 2026-02-13T02:23:36.205 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.200+0000 7f12cb7fe6c0 1 -- 10.20.193.27:0/2124265908 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f12d8002d00 con 0x7f12e010f200 2026-02-13T02:23:36.205 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.200+0000 7f12e89556c0 1 -- 10.20.193.27:0/2124265908 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e0163e20 con 0x7f12e010f200 2026-02-13T02:23:36.205 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.200+0000 7f12cb7fe6c0 1 -- 10.20.193.27:0/2124265908 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f12d80372f0 con 0x7f12e010f200 2026-02-13T02:23:36.206 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.200+0000 7f12e89556c0 1 -- 10.20.193.27:0/2124265908 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e0164250 con 0x7f12e010f200 2026-02-13T02:23:36.206 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.201+0000 7f12e89556c0 1 -- 10.20.193.27:0/2124265908 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12e01104d0 con 0x7f12e010f200 2026-02-13T02:23:36.207 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.202+0000 7f12cb7fe6c0 1 -- 10.20.193.27:0/2124265908 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f12d80078d0 con 0x7f12e010f200 2026-02-13T02:23:36.207 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.202+0000 7f12cb7fe6c0 1 --2- 10.20.193.27:0/2124265908 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f12bc0b5600 0x7f12bc0d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:36.207 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.203+0000 7f12cb7fe6c0 1 -- 10.20.193.27:0/2124265908 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f12d8107c20 con 0x7f12e010f200 2026-02-13T02:23:36.207 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.203+0000 7f12e5ec56c0 1 --2- 10.20.193.27:0/2124265908 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f12bc0b5600 0x7f12bc0d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:36.207 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.203+0000 7f12e5ec56c0 1 --2- 10.20.193.27:0/2124265908 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f12bc0b5600 0x7f12bc0d5ab0 secure :-1 s=READY pgs=250 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f12cc005980 tx=0x7f12cc005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:36.207 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.203+0000 7f12cb7fe6c0 1 -- 10.20.193.27:0/2124265908 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f12d80ca6a0 con 0x7f12e010f200 2026-02-13T02:23:36.320 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.319+0000 7f12e89556c0 1 -- 10.20.193.27:0/2124265908 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f12e010f5d0 con 0x7f12e010f200 2026-02-13T02:23:36.320 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:36 trial027 ceph-mon[24567]: pgmap v226: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2026-02-13T02:23:36.321 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.319+0000 7f12cb7fe6c0 1 -- 10.20.193.27:0/2124265908 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f12d80cd850 con 0x7f12e010f200 2026-02-13T02:23:36.321 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:23:36.321 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:23:36.322 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.320+0000 7f12e89556c0 1 -- 10.20.193.27:0/2124265908 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f12bc0b5600 msgr2=0x7f12bc0d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:36.322 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.320+0000 7f12e89556c0 1 --2- 10.20.193.27:0/2124265908 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f12bc0b5600 0x7f12bc0d5ab0 secure :-1 s=READY pgs=250 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f12cc005980 tx=0x7f12cc005910 comp rx=0 tx=0).stop 2026-02-13T02:23:36.322 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.320+0000 7f12e89556c0 1 -- 10.20.193.27:0/2124265908 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f12e010f200 msgr2=0x7f12e011d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:36.323 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.320+0000 7f12e89556c0 1 --2- 10.20.193.27:0/2124265908 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f12e010f200 0x7f12e011d390 secure :-1 s=READY pgs=373 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f12d802f230 tx=0x7f12d802fd40 comp rx=0 tx=0).stop 2026-02-13T02:23:36.323 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.320+0000 7f12e89556c0 1 -- 10.20.193.27:0/2124265908 shutdown_connections 2026-02-13T02:23:36.323 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.320+0000 7f12e89556c0 1 --2- 10.20.193.27:0/2124265908 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f12bc0b5600 0x7f12bc0d5ab0 unknown :-1 s=CLOSED pgs=250 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:36.323 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.320+0000 7f12e89556c0 1 --2- 10.20.193.27:0/2124265908 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f12e0122c90 0x7f12e0123130 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:36.323 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.320+0000 7f12e89556c0 1 --2- 10.20.193.27:0/2124265908 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f12e0115410 0x7f12e011d8d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:36.323 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.320+0000 7f12e89556c0 1 --2- 10.20.193.27:0/2124265908 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f12e010f200 0x7f12e011d390 unknown :-1 s=CLOSED pgs=373 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:36.323 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.320+0000 7f12e89556c0 1 -- 10.20.193.27:0/2124265908 >> 10.20.193.27:0/2124265908 conn(0x7f12e00ecb00 msgr2=0x7f12e010cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:36.323 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.320+0000 7f12e89556c0 1 -- 10.20.193.27:0/2124265908 shutdown_connections 2026-02-13T02:23:36.323 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:36.320+0000 7f12e89556c0 1 -- 10.20.193.27:0/2124265908 wait complete. 2026-02-13T02:23:36.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:36 trial028 ceph-mon[26701]: pgmap v226: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2026-02-13T02:23:36.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:36 trial107 ceph-mon[27422]: pgmap v226: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2026-02-13T02:23:37.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:37 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2124265908' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:37.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:37 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2124265908' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:37.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:37 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2124265908' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:38.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:38 trial028 ceph-mon[26701]: pgmap v227: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2026-02-13T02:23:38.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:38 trial107 ceph-mon[27422]: pgmap v227: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2026-02-13T02:23:38.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:38 trial027 ceph-mon[24567]: pgmap v227: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2026-02-13T02:23:39.361 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:23:39.480 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:23:39.596 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.592+0000 7fd45b6b36c0 1 -- 10.20.193.27:0/3019828490 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd45410fbf0 msgr2=0x7fd454114c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:39.597 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.592+0000 7fd45b6b36c0 1 --2- 10.20.193.27:0/3019828490 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd45410fbf0 0x7fd454114c30 secure :-1 s=READY pgs=374 gs=1 cs=0 l=1 c_cookie=17565039626454002033 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd4480099a0 tx=0x7fd44802ed20 comp rx=0 tx=0).stop 2026-02-13T02:23:39.597 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.592+0000 7fd45b6b36c0 1 -- 10.20.193.27:0/3019828490 shutdown_connections 2026-02-13T02:23:39.597 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.592+0000 7fd45b6b36c0 1 --2- 10.20.193.27:0/3019828490 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd454115410 0x7fd454175db0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:39.597 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.592+0000 7fd45b6b36c0 1 --2- 10.20.193.27:0/3019828490 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd45410fbf0 0x7fd454114c30 unknown :-1 s=CLOSED pgs=374 gs=1 cs=0 l=1 c_cookie=17565039626454002033 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:39.597 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.592+0000 7fd45b6b36c0 1 --2- 10.20.193.27:0/3019828490 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd45410f200 0x7fd45410f5d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:39.597 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.592+0000 7fd45b6b36c0 1 -- 10.20.193.27:0/3019828490 >> 10.20.193.27:0/3019828490 conn(0x7fd4540ecb00 msgr2=0x7fd45410cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:39.597 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.592+0000 7fd45b6b36c0 1 -- 10.20.193.27:0/3019828490 shutdown_connections 2026-02-13T02:23:39.597 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.592+0000 7fd45b6b36c0 1 -- 10.20.193.27:0/3019828490 wait complete. 2026-02-13T02:23:39.597 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.593+0000 7fd45b6b36c0 1 Processor -- start 2026-02-13T02:23:39.597 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.593+0000 7fd45b6b36c0 1 -- start start 2026-02-13T02:23:39.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.593+0000 7fd45b6b36c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd45410f200 0x7fd4540f4c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:39.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.593+0000 7fd45b6b36c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd454115410 0x7fd454175cb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:39.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.593+0000 7fd45b6b36c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd4540f52c0 0x7fd4540fa590 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:39.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.593+0000 7fd45b6b36c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd4540f59b0 con 0x7fd45410f200 2026-02-13T02:23:39.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.593+0000 7fd45b6b36c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd4540faad0 con 0x7fd454115410 2026-02-13T02:23:39.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.593+0000 7fd45b6b36c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd4540fac90 con 0x7fd4540f52c0 2026-02-13T02:23:39.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.593+0000 7fd4594246c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd45410f200 0x7fd4540f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:39.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.593+0000 7fd4594246c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd45410f200 0x7fd4540f4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:57470/0 (socket says 10.20.193.27:57470) 2026-02-13T02:23:39.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.593+0000 7fd4594246c0 1 -- 10.20.193.27:0/4065991233 learned_addr learned my addr 10.20.193.27:0/4065991233 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:23:39.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.593+0000 7fd4594246c0 1 -- 10.20.193.27:0/4065991233 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd4540f52c0 msgr2=0x7fd4540fa590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:39.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.593+0000 7fd4594246c0 1 --2- 10.20.193.27:0/4065991233 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd4540f52c0 0x7fd4540fa590 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:39.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.593+0000 7fd4594246c0 1 -- 10.20.193.27:0/4065991233 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd454115410 msgr2=0x7fd454175cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:39.599 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.593+0000 7fd458c236c0 1 --2- 10.20.193.27:0/4065991233 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd454115410 0x7fd454175cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:39.599 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.593+0000 7fd4594246c0 1 --2- 10.20.193.27:0/4065991233 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd454115410 0x7fd454175cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:39.599 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.593+0000 7fd4594246c0 1 -- 10.20.193.27:0/4065991233 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd448009590 con 0x7fd45410f200 2026-02-13T02:23:39.599 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.593+0000 7fd458c236c0 1 --2- 10.20.193.27:0/4065991233 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd454115410 0x7fd454175cb0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:23:39.599 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.593+0000 7fd4594246c0 1 --2- 10.20.193.27:0/4065991233 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd45410f200 0x7fd4540f4c90 secure :-1 s=READY pgs=375 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd44400b710 tx=0x7fd44400bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:39.599 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.593+0000 7fd4427fc6c0 1 -- 10.20.193.27:0/4065991233 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd444004250 con 0x7fd45410f200 2026-02-13T02:23:39.599 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.593+0000 7fd4427fc6c0 1 -- 10.20.193.27:0/4065991233 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd444002d80 con 0x7fd45410f200 2026-02-13T02:23:39.599 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.593+0000 7fd45b6b36c0 1 -- 10.20.193.27:0/4065991233 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4540faf70 con 0x7fd45410f200 2026-02-13T02:23:39.599 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.593+0000 7fd4427fc6c0 1 -- 10.20.193.27:0/4065991233 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd44400cda0 con 0x7fd45410f200 2026-02-13T02:23:39.599 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.593+0000 7fd45b6b36c0 1 -- 10.20.193.27:0/4065991233 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4540fb700 con 0x7fd45410f200 2026-02-13T02:23:39.599 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.594+0000 7fd45b6b36c0 1 -- 10.20.193.27:0/4065991233 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4541104d0 con 0x7fd45410f200 2026-02-13T02:23:39.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.595+0000 7fd4427fc6c0 1 -- 10.20.193.27:0/4065991233 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7fd4440028c0 con 0x7fd45410f200 2026-02-13T02:23:39.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.596+0000 7fd4427fc6c0 1 --2- 10.20.193.27:0/4065991233 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd4300b5680 0x7fd4300d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:39.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.596+0000 7fd458c236c0 1 --2- 10.20.193.27:0/4065991233 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd4300b5680 0x7fd4300d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:39.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.596+0000 7fd4427fc6c0 1 -- 10.20.193.27:0/4065991233 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7fd4440e3ad0 con 0x7fd45410f200 2026-02-13T02:23:39.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.596+0000 7fd4427fc6c0 1 -- 10.20.193.27:0/4065991233 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fd4440a6550 con 0x7fd45410f200 2026-02-13T02:23:39.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.596+0000 7fd458c236c0 1 --2- 10.20.193.27:0/4065991233 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd4300b5680 0x7fd4300d5b30 secure :-1 s=READY pgs=251 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd4480098c0 tx=0x7fd44802f580 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:39.712 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.710+0000 7fd45b6b36c0 1 -- 10.20.193.27:0/4065991233 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd45410f5d0 con 0x7fd45410f200 2026-02-13T02:23:39.712 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.710+0000 7fd4427fc6c0 1 -- 10.20.193.27:0/4065991233 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7fd4440a9700 con 0x7fd45410f200 2026-02-13T02:23:39.712 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:23:39.712 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:23:39.713 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.711+0000 7fd45b6b36c0 1 -- 10.20.193.27:0/4065991233 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd4300b5680 msgr2=0x7fd4300d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:39.713 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.711+0000 7fd45b6b36c0 1 --2- 10.20.193.27:0/4065991233 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd4300b5680 0x7fd4300d5b30 secure :-1 s=READY pgs=251 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd4480098c0 tx=0x7fd44802f580 comp rx=0 tx=0).stop 2026-02-13T02:23:39.714 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.711+0000 7fd45b6b36c0 1 -- 10.20.193.27:0/4065991233 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd45410f200 msgr2=0x7fd4540f4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:39.714 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.711+0000 7fd45b6b36c0 1 --2- 10.20.193.27:0/4065991233 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd45410f200 0x7fd4540f4c90 secure :-1 s=READY pgs=375 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd44400b710 tx=0x7fd44400bbe0 comp rx=0 tx=0).stop 2026-02-13T02:23:39.714 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.711+0000 7fd45b6b36c0 1 -- 10.20.193.27:0/4065991233 shutdown_connections 2026-02-13T02:23:39.714 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.711+0000 7fd45b6b36c0 1 --2- 10.20.193.27:0/4065991233 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd4300b5680 0x7fd4300d5b30 unknown :-1 s=CLOSED pgs=251 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:39.714 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.711+0000 7fd45b6b36c0 1 --2- 10.20.193.27:0/4065991233 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd4540f52c0 0x7fd4540fa590 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:39.714 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.711+0000 7fd45b6b36c0 1 --2- 10.20.193.27:0/4065991233 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd454115410 0x7fd454175cb0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:39.714 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.711+0000 7fd45b6b36c0 1 --2- 10.20.193.27:0/4065991233 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd45410f200 0x7fd4540f4c90 unknown :-1 s=CLOSED pgs=375 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:39.714 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.711+0000 7fd45b6b36c0 1 -- 10.20.193.27:0/4065991233 >> 10.20.193.27:0/4065991233 conn(0x7fd4540ecb00 msgr2=0x7fd454113d10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:39.714 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.711+0000 7fd45b6b36c0 1 -- 10.20.193.27:0/4065991233 shutdown_connections 2026-02-13T02:23:39.714 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:39.711+0000 7fd45b6b36c0 1 -- 10.20.193.27:0/4065991233 wait complete. 2026-02-13T02:23:40.465 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:40 trial028 ceph-mon[26701]: pgmap v228: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2026-02-13T02:23:40.465 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:40 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/4065991233' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:40.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:40 trial107 ceph-mon[27422]: pgmap v228: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2026-02-13T02:23:40.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:40 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/4065991233' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:40.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:40 trial027 ceph-mon[24567]: pgmap v228: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2026-02-13T02:23:40.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:40 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/4065991233' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:42.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:42 trial028 ceph-mon[26701]: pgmap v229: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2026-02-13T02:23:42.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:42 trial107 ceph-mon[27422]: pgmap v229: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2026-02-13T02:23:42.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:42 trial027 ceph-mon[24567]: pgmap v229: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2026-02-13T02:23:42.754 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:23:42.875 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:23:42.992 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.987+0000 7f05f29a96c0 1 -- 10.20.193.27:0/3562774645 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f05ec10fbf0 msgr2=0x7f05ec114c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:42.992 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.987+0000 7f05f29a96c0 1 --2- 10.20.193.27:0/3562774645 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f05ec10fbf0 0x7f05ec114c30 secure :-1 s=READY pgs=376 gs=1 cs=0 l=1 c_cookie=9272033916082780253 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f05dc0099a0 tx=0x7f05dc02ed20 comp rx=0 tx=0).stop 2026-02-13T02:23:42.992 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.987+0000 7f05f29a96c0 1 -- 10.20.193.27:0/3562774645 shutdown_connections 2026-02-13T02:23:42.992 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.987+0000 7f05f29a96c0 1 --2- 10.20.193.27:0/3562774645 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f05ec115410 0x7f05ec175db0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:42.992 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.987+0000 7f05f29a96c0 1 --2- 10.20.193.27:0/3562774645 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f05ec10fbf0 0x7f05ec114c30 unknown :-1 s=CLOSED pgs=376 gs=1 cs=0 l=1 c_cookie=9272033916082780253 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:42.992 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.987+0000 7f05f29a96c0 1 --2- 10.20.193.27:0/3562774645 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f05ec10f200 0x7f05ec10f5d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:42.993 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.987+0000 7f05f29a96c0 1 -- 10.20.193.27:0/3562774645 >> 10.20.193.27:0/3562774645 conn(0x7f05ec0ecb00 msgr2=0x7f05ec10cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:42.993 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.987+0000 7f05f29a96c0 1 -- 10.20.193.27:0/3562774645 shutdown_connections 2026-02-13T02:23:42.993 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.987+0000 7f05f29a96c0 1 -- 10.20.193.27:0/3562774645 wait complete. 2026-02-13T02:23:42.993 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.987+0000 7f05f29a96c0 1 Processor -- start 2026-02-13T02:23:42.993 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.987+0000 7f05f29a96c0 1 -- start start 2026-02-13T02:23:42.993 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.988+0000 7f05f29a96c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f05ec10f200 0x7f05ec0f4ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:42.993 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.988+0000 7f05f29a96c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f05ec115410 0x7f05ec175cb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:42.993 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.988+0000 7f05f29a96c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f05ec0f5310 0x7f05ec0fa5e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:42.993 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.988+0000 7f05f29a96c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec0f5a00 con 0x7f05ec10f200 2026-02-13T02:23:42.993 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.988+0000 7f05f29a96c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec0fab20 con 0x7f05ec115410 2026-02-13T02:23:42.993 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.988+0000 7f05f29a96c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec0face0 con 0x7f05ec0f5310 2026-02-13T02:23:42.993 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.988+0000 7f05ebfff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f05ec10f200 0x7f05ec0f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:42.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.988+0000 7f05ebfff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f05ec10f200 0x7f05ec0f4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:57478/0 (socket says 10.20.193.27:57478) 2026-02-13T02:23:42.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.988+0000 7f05ebfff6c0 1 -- 10.20.193.27:0/449769608 learned_addr learned my addr 10.20.193.27:0/449769608 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:23:42.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.988+0000 7f05eb7fe6c0 1 --2- 10.20.193.27:0/449769608 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f05ec115410 0x7f05ec175cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:42.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.988+0000 7f05ebfff6c0 1 -- 10.20.193.27:0/449769608 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f05ec0f5310 msgr2=0x7f05ec0fa5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:42.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.988+0000 7f05ebfff6c0 1 --2- 10.20.193.27:0/449769608 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f05ec0f5310 0x7f05ec0fa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:42.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.988+0000 7f05ebfff6c0 1 -- 10.20.193.27:0/449769608 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f05ec115410 msgr2=0x7f05ec175cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:42.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.988+0000 7f05ebfff6c0 1 --2- 10.20.193.27:0/449769608 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f05ec115410 0x7f05ec175cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:42.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.988+0000 7f05ebfff6c0 1 -- 10.20.193.27:0/449769608 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05dc009590 con 0x7f05ec10f200 2026-02-13T02:23:42.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.988+0000 7f05eb7fe6c0 1 --2- 10.20.193.27:0/449769608 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f05ec115410 0x7f05ec175cb0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:23:42.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.988+0000 7f05ebfff6c0 1 --2- 10.20.193.27:0/449769608 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f05ec10f200 0x7f05ec0f4ce0 secure :-1 s=READY pgs=377 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f05d800b710 tx=0x7f05d800bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:42.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.988+0000 7f05e97fa6c0 1 -- 10.20.193.27:0/449769608 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f05d8004250 con 0x7f05ec10f200 2026-02-13T02:23:42.994 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.988+0000 7f05e97fa6c0 1 -- 10.20.193.27:0/449769608 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f05d8002d80 con 0x7f05ec10f200 2026-02-13T02:23:42.995 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.988+0000 7f05f29a96c0 1 -- 10.20.193.27:0/449769608 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05ec0fafc0 con 0x7f05ec10f200 2026-02-13T02:23:42.995 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.988+0000 7f05e97fa6c0 1 -- 10.20.193.27:0/449769608 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f05d800cda0 con 0x7f05ec10f200 2026-02-13T02:23:42.995 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.988+0000 7f05f29a96c0 1 -- 10.20.193.27:0/449769608 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05ec0fb750 con 0x7f05ec10f200 2026-02-13T02:23:42.995 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.988+0000 7f05f29a96c0 1 -- 10.20.193.27:0/449769608 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05ec1104d0 con 0x7f05ec10f200 2026-02-13T02:23:42.995 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.990+0000 7f05e97fa6c0 1 -- 10.20.193.27:0/449769608 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f05d80028c0 con 0x7f05ec10f200 2026-02-13T02:23:42.996 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.990+0000 7f05e97fa6c0 1 --2- 10.20.193.27:0/449769608 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f05c00b5680 0x7f05c00d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:42.996 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.990+0000 7f05e97fa6c0 1 -- 10.20.193.27:0/449769608 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f05d80e3ad0 con 0x7f05ec10f200 2026-02-13T02:23:42.996 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.990+0000 7f05eb7fe6c0 1 --2- 10.20.193.27:0/449769608 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f05c00b5680 0x7f05c00d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:42.996 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.991+0000 7f05eb7fe6c0 1 --2- 10.20.193.27:0/449769608 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f05c00b5680 0x7f05c00d5b30 secure :-1 s=READY pgs=252 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f05dc0098c0 tx=0x7f05dc009250 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:42.996 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:42.991+0000 7f05e97fa6c0 1 -- 10.20.193.27:0/449769608 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f05d80a6550 con 0x7f05ec10f200 2026-02-13T02:23:43.110 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:43.107+0000 7f05f29a96c0 1 -- 10.20.193.27:0/449769608 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f05ec10f5d0 con 0x7f05ec10f200 2026-02-13T02:23:43.110 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:43.107+0000 7f05e97fa6c0 1 -- 10.20.193.27:0/449769608 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f05d80a9700 con 0x7f05ec10f200 2026-02-13T02:23:43.110 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:23:43.110 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:23:43.111 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:43.108+0000 7f05f29a96c0 1 -- 10.20.193.27:0/449769608 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f05c00b5680 msgr2=0x7f05c00d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:43.111 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:43.108+0000 7f05f29a96c0 1 --2- 10.20.193.27:0/449769608 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f05c00b5680 0x7f05c00d5b30 secure :-1 s=READY pgs=252 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f05dc0098c0 tx=0x7f05dc009250 comp rx=0 tx=0).stop 2026-02-13T02:23:43.111 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:43.108+0000 7f05f29a96c0 1 -- 10.20.193.27:0/449769608 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f05ec10f200 msgr2=0x7f05ec0f4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:43.112 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:43.108+0000 7f05f29a96c0 1 --2- 10.20.193.27:0/449769608 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f05ec10f200 0x7f05ec0f4ce0 secure :-1 s=READY pgs=377 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f05d800b710 tx=0x7f05d800bbe0 comp rx=0 tx=0).stop 2026-02-13T02:23:43.112 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:43.108+0000 7f05f29a96c0 1 -- 10.20.193.27:0/449769608 shutdown_connections 2026-02-13T02:23:43.112 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:43.108+0000 7f05f29a96c0 1 --2- 10.20.193.27:0/449769608 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f05c00b5680 0x7f05c00d5b30 unknown :-1 s=CLOSED pgs=252 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:43.112 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:43.108+0000 7f05f29a96c0 1 --2- 10.20.193.27:0/449769608 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f05ec0f5310 0x7f05ec0fa5e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:43.112 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:43.108+0000 7f05f29a96c0 1 --2- 10.20.193.27:0/449769608 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f05ec115410 0x7f05ec175cb0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:43.112 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:43.108+0000 7f05f29a96c0 1 --2- 10.20.193.27:0/449769608 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f05ec10f200 0x7f05ec0f4ce0 unknown :-1 s=CLOSED pgs=377 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:43.112 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:43.108+0000 7f05f29a96c0 1 -- 10.20.193.27:0/449769608 >> 10.20.193.27:0/449769608 conn(0x7f05ec0ecb00 msgr2=0x7f05ec10d900 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:43.112 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:43.108+0000 7f05f29a96c0 1 -- 10.20.193.27:0/449769608 shutdown_connections 2026-02-13T02:23:43.112 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:43.108+0000 7f05f29a96c0 1 -- 10.20.193.27:0/449769608 wait complete. 2026-02-13T02:23:43.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:43 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/449769608' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:43.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:43 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/449769608' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:43.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:43 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/449769608' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:44.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:44 trial028 ceph-mon[26701]: pgmap v230: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2026-02-13T02:23:44.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:44 trial107 ceph-mon[27422]: pgmap v230: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2026-02-13T02:23:44.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:44 trial027 ceph-mon[24567]: pgmap v230: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2026-02-13T02:23:46.154 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:23:46.273 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:23:46.401 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.397+0000 7f29e3c576c0 1 -- 10.20.193.27:0/2734756767 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f29dc10f200 msgr2=0x7f29dc10f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:46.402 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.397+0000 7f29e3c576c0 1 --2- 10.20.193.27:0/2734756767 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f29dc10f200 0x7f29dc10f5d0 secure :-1 s=READY pgs=378 gs=1 cs=0 l=1 c_cookie=8587663800865324007 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f29cc0099a0 tx=0x7f29cc02ed20 comp rx=0 tx=0).stop 2026-02-13T02:23:46.402 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.397+0000 7f29e3c576c0 1 -- 10.20.193.27:0/2734756767 shutdown_connections 2026-02-13T02:23:46.402 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.397+0000 7f29e3c576c0 1 --2- 10.20.193.27:0/2734756767 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f29dc115410 0x7f29dc175db0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:46.402 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.397+0000 7f29e3c576c0 1 --2- 10.20.193.27:0/2734756767 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f29dc10fbf0 0x7f29dc114c30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:46.402 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.397+0000 7f29e3c576c0 1 --2- 10.20.193.27:0/2734756767 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f29dc10f200 0x7f29dc10f5d0 unknown :-1 s=CLOSED pgs=378 gs=1 cs=0 l=1 c_cookie=8587663800865324007 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:46.402 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.397+0000 7f29e3c576c0 1 -- 10.20.193.27:0/2734756767 >> 10.20.193.27:0/2734756767 conn(0x7f29dc0ecb00 msgr2=0x7f29dc10cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:46.402 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.397+0000 7f29e3c576c0 1 -- 10.20.193.27:0/2734756767 shutdown_connections 2026-02-13T02:23:46.402 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.397+0000 7f29e3c576c0 1 -- 10.20.193.27:0/2734756767 wait complete. 2026-02-13T02:23:46.402 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.397+0000 7f29e3c576c0 1 Processor -- start 2026-02-13T02:23:46.403 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.397+0000 7f29e3c576c0 1 -- start start 2026-02-13T02:23:46.403 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.397+0000 7f29e3c576c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f29dc10f200 0x7f29dc11d390 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:46.403 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.398+0000 7f29e3c576c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f29dc115410 0x7f29dc11d8d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:46.403 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.398+0000 7f29e3c576c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f29dc122c90 0x7f29dc123130 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:46.403 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.398+0000 7f29e3c576c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f29dc1236a0 con 0x7f29dc122c90 2026-02-13T02:23:46.403 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.398+0000 7f29e3c576c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f29dc123810 con 0x7f29dc115410 2026-02-13T02:23:46.403 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.398+0000 7f29e3c576c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f29dc1239d0 con 0x7f29dc10f200 2026-02-13T02:23:46.403 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.398+0000 7f29e21c96c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f29dc122c90 0x7f29dc123130 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:46.403 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.398+0000 7f29e21c96c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f29dc122c90 0x7f29dc123130 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:57494/0 (socket says 10.20.193.27:57494) 2026-02-13T02:23:46.403 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.398+0000 7f29e21c96c0 1 -- 10.20.193.27:0/3693677419 learned_addr learned my addr 10.20.193.27:0/3693677419 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:23:46.403 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.398+0000 7f29e19c86c0 1 --2- 10.20.193.27:0/3693677419 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f29dc10f200 0x7f29dc11d390 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:46.403 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.398+0000 7f29e11c76c0 1 --2- 10.20.193.27:0/3693677419 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f29dc115410 0x7f29dc11d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:46.404 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.398+0000 7f29e21c96c0 1 -- 10.20.193.27:0/3693677419 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f29dc10f200 msgr2=0x7f29dc11d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:46.404 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.398+0000 7f29e21c96c0 1 --2- 10.20.193.27:0/3693677419 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f29dc10f200 0x7f29dc11d390 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:46.404 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.398+0000 7f29e21c96c0 1 -- 10.20.193.27:0/3693677419 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f29dc115410 msgr2=0x7f29dc11d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:46.404 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.398+0000 7f29e21c96c0 1 --2- 10.20.193.27:0/3693677419 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f29dc115410 0x7f29dc11d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:46.404 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.398+0000 7f29e21c96c0 1 -- 10.20.193.27:0/3693677419 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29cc009590 con 0x7f29dc122c90 2026-02-13T02:23:46.404 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.398+0000 7f29e11c76c0 1 --2- 10.20.193.27:0/3693677419 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f29dc115410 0x7f29dc11d8d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:23:46.404 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.398+0000 7f29e21c96c0 1 --2- 10.20.193.27:0/3693677419 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f29dc122c90 0x7f29dc123130 secure :-1 s=READY pgs=379 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f29d8006ea0 tx=0x7f29d800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:46.404 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.398+0000 7f29caffd6c0 1 -- 10.20.193.27:0/3693677419 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f29d8003a30 con 0x7f29dc122c90 2026-02-13T02:23:46.404 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.398+0000 7f29caffd6c0 1 -- 10.20.193.27:0/3693677419 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f29d80054f0 con 0x7f29dc122c90 2026-02-13T02:23:46.404 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.398+0000 7f29e3c576c0 1 -- 10.20.193.27:0/3693677419 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29dc163e80 con 0x7f29dc122c90 2026-02-13T02:23:46.404 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.398+0000 7f29caffd6c0 1 -- 10.20.193.27:0/3693677419 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f29d800dcd0 con 0x7f29dc122c90 2026-02-13T02:23:46.404 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.398+0000 7f29e3c576c0 1 -- 10.20.193.27:0/3693677419 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29dc164310 con 0x7f29dc122c90 2026-02-13T02:23:46.405 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.398+0000 7f29e3c576c0 1 -- 10.20.193.27:0/3693677419 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29dc1104d0 con 0x7f29dc122c90 2026-02-13T02:23:46.405 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.400+0000 7f29caffd6c0 1 -- 10.20.193.27:0/3693677419 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f29d8005030 con 0x7f29dc122c90 2026-02-13T02:23:46.406 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.400+0000 7f29caffd6c0 1 --2- 10.20.193.27:0/3693677419 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f29b80b5600 0x7f29b80d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:46.406 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.400+0000 7f29e19c86c0 1 --2- 10.20.193.27:0/3693677419 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f29b80b5600 0x7f29b80d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:46.406 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.400+0000 7f29caffd6c0 1 -- 10.20.193.27:0/3693677419 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f29d80e4b50 con 0x7f29dc122c90 2026-02-13T02:23:46.406 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.401+0000 7f29caffd6c0 1 -- 10.20.193.27:0/3693677419 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f29d811d5e0 con 0x7f29dc122c90 2026-02-13T02:23:46.406 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.401+0000 7f29e19c86c0 1 --2- 10.20.193.27:0/3693677419 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f29b80b5600 0x7f29b80d5ab0 secure :-1 s=READY pgs=253 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f29cc0092e0 tx=0x7f29cc005c00 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:46.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:46 trial028 ceph-mon[26701]: pgmap v231: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:23:46.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:46 trial107 ceph-mon[27422]: pgmap v231: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:23:46.517 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.515+0000 7f29e3c576c0 1 -- 10.20.193.27:0/3693677419 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f29dc11e4e0 con 0x7f29dc122c90 2026-02-13T02:23:46.517 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:46 trial027 ceph-mon[24567]: pgmap v231: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:23:46.518 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.515+0000 7f29caffd6c0 1 -- 10.20.193.27:0/3693677419 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f29d80a75d0 con 0x7f29dc122c90 2026-02-13T02:23:46.518 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:23:46.518 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:23:46.519 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.516+0000 7f29e3c576c0 1 -- 10.20.193.27:0/3693677419 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f29b80b5600 msgr2=0x7f29b80d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:46.519 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.516+0000 7f29e3c576c0 1 --2- 10.20.193.27:0/3693677419 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f29b80b5600 0x7f29b80d5ab0 secure :-1 s=READY pgs=253 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f29cc0092e0 tx=0x7f29cc005c00 comp rx=0 tx=0).stop 2026-02-13T02:23:46.519 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.516+0000 7f29e3c576c0 1 -- 10.20.193.27:0/3693677419 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f29dc122c90 msgr2=0x7f29dc123130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:46.519 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.516+0000 7f29e3c576c0 1 --2- 10.20.193.27:0/3693677419 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f29dc122c90 0x7f29dc123130 secure :-1 s=READY pgs=379 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f29d8006ea0 tx=0x7f29d800cca0 comp rx=0 tx=0).stop 2026-02-13T02:23:46.519 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.516+0000 7f29e3c576c0 1 -- 10.20.193.27:0/3693677419 shutdown_connections 2026-02-13T02:23:46.520 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.516+0000 7f29e3c576c0 1 --2- 10.20.193.27:0/3693677419 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f29b80b5600 0x7f29b80d5ab0 unknown :-1 s=CLOSED pgs=253 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:46.520 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.516+0000 7f29e3c576c0 1 --2- 10.20.193.27:0/3693677419 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f29dc122c90 0x7f29dc123130 unknown :-1 s=CLOSED pgs=379 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:46.520 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.516+0000 7f29e3c576c0 1 --2- 10.20.193.27:0/3693677419 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f29dc115410 0x7f29dc11d8d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:46.520 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.516+0000 7f29e3c576c0 1 --2- 10.20.193.27:0/3693677419 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f29dc10f200 0x7f29dc11d390 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:46.520 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.516+0000 7f29e3c576c0 1 -- 10.20.193.27:0/3693677419 >> 10.20.193.27:0/3693677419 conn(0x7f29dc0ecb00 msgr2=0x7f29dc10cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:46.520 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.516+0000 7f29e3c576c0 1 -- 10.20.193.27:0/3693677419 shutdown_connections 2026-02-13T02:23:46.520 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:46.516+0000 7f29e3c576c0 1 -- 10.20.193.27:0/3693677419 wait complete. 2026-02-13T02:23:47.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:47 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3693677419' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:47.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:47 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3693677419' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:47.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:47 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3693677419' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:48.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:48 trial028 ceph-mon[26701]: pgmap v232: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:23:48.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:48 trial107 ceph-mon[27422]: pgmap v232: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:23:48.559 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:48 trial027 ceph-mon[24567]: pgmap v232: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:23:49.559 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:23:49.675 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:23:49.792 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.788+0000 7f2e740646c0 1 -- 10.20.193.27:0/349960629 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2e6c10cfb0 msgr2=0x7f2e6c10d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:49.792 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.788+0000 7f2e740646c0 1 --2- 10.20.193.27:0/349960629 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2e6c10cfb0 0x7f2e6c10d380 secure :-1 s=READY pgs=380 gs=1 cs=0 l=1 c_cookie=5403180079138502766 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2e5c0099a0 tx=0x7f2e5c02ed20 comp rx=0 tx=0).stop 2026-02-13T02:23:49.793 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.788+0000 7f2e740646c0 1 -- 10.20.193.27:0/349960629 shutdown_connections 2026-02-13T02:23:49.793 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.788+0000 7f2e740646c0 1 --2- 10.20.193.27:0/349960629 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2e6c1131c0 0x7f2e6c173b60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:49.793 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.788+0000 7f2e740646c0 1 --2- 10.20.193.27:0/349960629 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2e6c10d9a0 0x7f2e6c1129e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:49.793 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.788+0000 7f2e740646c0 1 --2- 10.20.193.27:0/349960629 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2e6c10cfb0 0x7f2e6c10d380 unknown :-1 s=CLOSED pgs=380 gs=1 cs=0 l=1 c_cookie=5403180079138502766 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:49.793 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.788+0000 7f2e740646c0 1 -- 10.20.193.27:0/349960629 >> 10.20.193.27:0/349960629 conn(0x7f2e6c0ea8b0 msgr2=0x7f2e6c10acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:49.793 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.788+0000 7f2e740646c0 1 -- 10.20.193.27:0/349960629 shutdown_connections 2026-02-13T02:23:49.793 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.788+0000 7f2e740646c0 1 -- 10.20.193.27:0/349960629 wait complete. 2026-02-13T02:23:49.793 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.789+0000 7f2e740646c0 1 Processor -- start 2026-02-13T02:23:49.793 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.789+0000 7f2e740646c0 1 -- start start 2026-02-13T02:23:49.793 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.789+0000 7f2e740646c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2e6c10cfb0 0x7f2e6c11b030 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:49.794 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.789+0000 7f2e740646c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2e6c1131c0 0x7f2e6c11b570 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:49.794 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.789+0000 7f2e740646c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2e6c120930 0x7f2e6c120dd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:49.794 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.789+0000 7f2e740646c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6c121340 con 0x7f2e6c120930 2026-02-13T02:23:49.794 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.789+0000 7f2e740646c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6c1214b0 con 0x7f2e6c1131c0 2026-02-13T02:23:49.794 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.789+0000 7f2e740646c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6c121670 con 0x7f2e6c10cfb0 2026-02-13T02:23:49.794 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.789+0000 7f2e725d66c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2e6c120930 0x7f2e6c120dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:49.794 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.789+0000 7f2e725d66c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2e6c120930 0x7f2e6c120dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:34090/0 (socket says 10.20.193.27:34090) 2026-02-13T02:23:49.794 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.789+0000 7f2e725d66c0 1 -- 10.20.193.27:0/114020391 learned_addr learned my addr 10.20.193.27:0/114020391 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:23:49.794 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.789+0000 7f2e71dd56c0 1 --2- 10.20.193.27:0/114020391 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2e6c10cfb0 0x7f2e6c11b030 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:49.794 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.789+0000 7f2e725d66c0 1 -- 10.20.193.27:0/114020391 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2e6c10cfb0 msgr2=0x7f2e6c11b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:49.794 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.789+0000 7f2e725d66c0 1 --2- 10.20.193.27:0/114020391 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2e6c10cfb0 0x7f2e6c11b030 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:49.794 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.789+0000 7f2e725d66c0 1 -- 10.20.193.27:0/114020391 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2e6c1131c0 msgr2=0x7f2e6c11b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:49.795 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.789+0000 7f2e725d66c0 1 --2- 10.20.193.27:0/114020391 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2e6c1131c0 0x7f2e6c11b570 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:49.795 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.789+0000 7f2e725d66c0 1 -- 10.20.193.27:0/114020391 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e5c009590 con 0x7f2e6c120930 2026-02-13T02:23:49.795 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.789+0000 7f2e71dd56c0 1 --2- 10.20.193.27:0/114020391 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2e6c10cfb0 0x7f2e6c11b030 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:23:49.795 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.789+0000 7f2e725d66c0 1 --2- 10.20.193.27:0/114020391 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2e6c120930 0x7f2e6c120dd0 secure :-1 s=READY pgs=381 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2e68006ea0 tx=0x7f2e6800ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:49.795 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.789+0000 7f2e5affd6c0 1 -- 10.20.193.27:0/114020391 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2e68003c40 con 0x7f2e6c120930 2026-02-13T02:23:49.795 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.789+0000 7f2e740646c0 1 -- 10.20.193.27:0/114020391 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e6c161b20 con 0x7f2e6c120930 2026-02-13T02:23:49.795 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.789+0000 7f2e5affd6c0 1 -- 10.20.193.27:0/114020391 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f2e68003da0 con 0x7f2e6c120930 2026-02-13T02:23:49.795 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.789+0000 7f2e5affd6c0 1 -- 10.20.193.27:0/114020391 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2e680081c0 con 0x7f2e6c120930 2026-02-13T02:23:49.795 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.789+0000 7f2e740646c0 1 -- 10.20.193.27:0/114020391 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e6c161fb0 con 0x7f2e6c120930 2026-02-13T02:23:49.795 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.790+0000 7f2e740646c0 1 -- 10.20.193.27:0/114020391 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e6c10e280 con 0x7f2e6c120930 2026-02-13T02:23:49.796 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.792+0000 7f2e5affd6c0 1 -- 10.20.193.27:0/114020391 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f2e68005860 con 0x7f2e6c120930 2026-02-13T02:23:49.796 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.792+0000 7f2e5affd6c0 1 --2- 10.20.193.27:0/114020391 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f2e480b5600 0x7f2e480d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:49.797 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.792+0000 7f2e71dd56c0 1 --2- 10.20.193.27:0/114020391 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f2e480b5600 0x7f2e480d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:49.797 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.793+0000 7f2e5affd6c0 1 -- 10.20.193.27:0/114020391 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f2e680e4bd0 con 0x7f2e6c120930 2026-02-13T02:23:49.797 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.793+0000 7f2e71dd56c0 1 --2- 10.20.193.27:0/114020391 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f2e480b5600 0x7f2e480d5ab0 secure :-1 s=READY pgs=254 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2e5c005d10 tx=0x7f2e5c005ca0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:49.797 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.793+0000 7f2e5affd6c0 1 -- 10.20.193.27:0/114020391 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f2e680ab0b0 con 0x7f2e6c120930 2026-02-13T02:23:49.909 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.906+0000 7f2e740646c0 1 -- 10.20.193.27:0/114020391 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2e6c10d380 con 0x7f2e6c120930 2026-02-13T02:23:49.909 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.906+0000 7f2e5affd6c0 1 -- 10.20.193.27:0/114020391 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f2e68071680 con 0x7f2e6c120930 2026-02-13T02:23:49.909 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:23:49.909 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:23:49.910 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.907+0000 7f2e740646c0 1 -- 10.20.193.27:0/114020391 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f2e480b5600 msgr2=0x7f2e480d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:49.910 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.907+0000 7f2e740646c0 1 --2- 10.20.193.27:0/114020391 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f2e480b5600 0x7f2e480d5ab0 secure :-1 s=READY pgs=254 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2e5c005d10 tx=0x7f2e5c005ca0 comp rx=0 tx=0).stop 2026-02-13T02:23:49.910 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.907+0000 7f2e740646c0 1 -- 10.20.193.27:0/114020391 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2e6c120930 msgr2=0x7f2e6c120dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:49.911 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.907+0000 7f2e740646c0 1 --2- 10.20.193.27:0/114020391 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2e6c120930 0x7f2e6c120dd0 secure :-1 s=READY pgs=381 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2e68006ea0 tx=0x7f2e6800ce50 comp rx=0 tx=0).stop 2026-02-13T02:23:49.911 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.907+0000 7f2e740646c0 1 -- 10.20.193.27:0/114020391 shutdown_connections 2026-02-13T02:23:49.911 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.907+0000 7f2e740646c0 1 --2- 10.20.193.27:0/114020391 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f2e480b5600 0x7f2e480d5ab0 unknown :-1 s=CLOSED pgs=254 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:49.911 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.907+0000 7f2e740646c0 1 --2- 10.20.193.27:0/114020391 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2e6c120930 0x7f2e6c120dd0 unknown :-1 s=CLOSED pgs=381 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:49.911 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.907+0000 7f2e740646c0 1 --2- 10.20.193.27:0/114020391 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2e6c1131c0 0x7f2e6c11b570 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:49.911 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.907+0000 7f2e740646c0 1 --2- 10.20.193.27:0/114020391 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2e6c10cfb0 0x7f2e6c11b030 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:49.911 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.907+0000 7f2e740646c0 1 -- 10.20.193.27:0/114020391 >> 10.20.193.27:0/114020391 conn(0x7f2e6c0ea8b0 msgr2=0x7f2e6c10ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:49.911 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.907+0000 7f2e740646c0 1 -- 10.20.193.27:0/114020391 shutdown_connections 2026-02-13T02:23:49.911 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:49.907+0000 7f2e740646c0 1 -- 10.20.193.27:0/114020391 wait complete. 2026-02-13T02:23:50.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:50 trial028 ceph-mon[26701]: pgmap v233: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:23:50.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:50 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/114020391' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:50.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:50 trial107 ceph-mon[27422]: pgmap v233: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:23:50.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:50 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/114020391' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:50.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:50 trial027 ceph-mon[24567]: pgmap v233: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:23:50.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:50 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/114020391' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:52.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:52 trial028 ceph-mon[26701]: pgmap v234: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:23:52.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:52 trial107 ceph-mon[27422]: pgmap v234: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:23:52.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:52 trial027 ceph-mon[24567]: pgmap v234: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:23:52.951 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:23:53.068 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:23:53.189 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.185+0000 7f5984fe86c0 1 -- 10.20.193.27:0/228122044 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f598010fc20 msgr2=0x7f5980114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:53.189 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.185+0000 7f5984fe86c0 1 --2- 10.20.193.27:0/228122044 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f598010fc20 0x7f5980114c60 secure :-1 s=READY pgs=382 gs=1 cs=0 l=1 c_cookie=5669914322173036265 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f59680099a0 tx=0x7f596802ed20 comp rx=0 tx=0).stop 2026-02-13T02:23:53.189 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.185+0000 7f5984fe86c0 1 -- 10.20.193.27:0/228122044 shutdown_connections 2026-02-13T02:23:53.190 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.185+0000 7f5984fe86c0 1 --2- 10.20.193.27:0/228122044 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5980115440 0x7f5980175de0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:53.190 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.185+0000 7f5984fe86c0 1 --2- 10.20.193.27:0/228122044 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f598010fc20 0x7f5980114c60 unknown :-1 s=CLOSED pgs=382 gs=1 cs=0 l=1 c_cookie=5669914322173036265 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:53.190 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.185+0000 7f5984fe86c0 1 --2- 10.20.193.27:0/228122044 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f598010f230 0x7f598010f600 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:53.190 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.185+0000 7f5984fe86c0 1 -- 10.20.193.27:0/228122044 >> 10.20.193.27:0/228122044 conn(0x7f59800ecb30 msgr2=0x7f598010cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:53.190 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.185+0000 7f5984fe86c0 1 -- 10.20.193.27:0/228122044 shutdown_connections 2026-02-13T02:23:53.190 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.185+0000 7f5984fe86c0 1 -- 10.20.193.27:0/228122044 wait complete. 2026-02-13T02:23:53.190 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.185+0000 7f5984fe86c0 1 Processor -- start 2026-02-13T02:23:53.190 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.185+0000 7f5984fe86c0 1 -- start start 2026-02-13T02:23:53.190 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.185+0000 7f5984fe86c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f598010f230 0x7f59800f4c60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:53.190 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.185+0000 7f5984fe86c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5980115440 0x7f5980175ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:53.190 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.185+0000 7f5984fe86c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f59800f5290 0x7f59800fa560 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:53.190 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.185+0000 7f5984fe86c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f59800f5980 con 0x7f5980115440 2026-02-13T02:23:53.191 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.185+0000 7f5984fe86c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f59800faaa0 con 0x7f59800f5290 2026-02-13T02:23:53.191 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.185+0000 7f5984fe86c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f59800fac60 con 0x7f598010f230 2026-02-13T02:23:53.191 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.185+0000 7f597dd746c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5980115440 0x7f5980175ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:53.191 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.185+0000 7f597dd746c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5980115440 0x7f5980175ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:34100/0 (socket says 10.20.193.27:34100) 2026-02-13T02:23:53.191 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.185+0000 7f597dd746c0 1 -- 10.20.193.27:0/97063904 learned_addr learned my addr 10.20.193.27:0/97063904 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:23:53.191 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.185+0000 7f597e5756c0 1 --2- 10.20.193.27:0/97063904 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f598010f230 0x7f59800f4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:53.191 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.186+0000 7f597dd746c0 1 -- 10.20.193.27:0/97063904 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f598010f230 msgr2=0x7f59800f4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:53.191 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.186+0000 7f597dd746c0 1 --2- 10.20.193.27:0/97063904 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f598010f230 0x7f59800f4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:53.191 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.186+0000 7f597ed766c0 1 --2- 10.20.193.27:0/97063904 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f59800f5290 0x7f59800fa560 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:53.191 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.186+0000 7f597dd746c0 1 -- 10.20.193.27:0/97063904 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f59800f5290 msgr2=0x7f59800fa560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:53.191 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.186+0000 7f597dd746c0 1 --2- 10.20.193.27:0/97063904 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f59800f5290 0x7f59800fa560 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:53.191 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.186+0000 7f597dd746c0 1 -- 10.20.193.27:0/97063904 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5968009590 con 0x7f5980115440 2026-02-13T02:23:53.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.186+0000 7f597ed766c0 1 --2- 10.20.193.27:0/97063904 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f59800f5290 0x7f59800fa560 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:23:53.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.186+0000 7f597dd746c0 1 --2- 10.20.193.27:0/97063904 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5980115440 0x7f5980175ce0 secure :-1 s=READY pgs=383 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5968004830 tx=0x7f596802fbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:53.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.186+0000 7f59677fe6c0 1 -- 10.20.193.27:0/97063904 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5968002ae0 con 0x7f5980115440 2026-02-13T02:23:53.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.186+0000 7f59677fe6c0 1 -- 10.20.193.27:0/97063904 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5968002c40 con 0x7f5980115440 2026-02-13T02:23:53.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.186+0000 7f5984fe86c0 1 -- 10.20.193.27:0/97063904 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59800faee0 con 0x7f5980115440 2026-02-13T02:23:53.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.186+0000 7f59677fe6c0 1 -- 10.20.193.27:0/97063904 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f59680362f0 con 0x7f5980115440 2026-02-13T02:23:53.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.186+0000 7f5984fe86c0 1 -- 10.20.193.27:0/97063904 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59800fb330 con 0x7f5980115440 2026-02-13T02:23:53.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.186+0000 7f5984fe86c0 1 -- 10.20.193.27:0/97063904 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5980110500 con 0x7f5980115440 2026-02-13T02:23:53.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.188+0000 7f59677fe6c0 1 -- 10.20.193.27:0/97063904 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f5968007810 con 0x7f5980115440 2026-02-13T02:23:53.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.188+0000 7f59677fe6c0 1 --2- 10.20.193.27:0/97063904 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f59580b5600 0x7f59580d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:53.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.189+0000 7f597e5756c0 1 --2- 10.20.193.27:0/97063904 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f59580b5600 0x7f59580d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:53.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.189+0000 7f59677fe6c0 1 -- 10.20.193.27:0/97063904 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f5968107d00 con 0x7f5980115440 2026-02-13T02:23:53.194 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.189+0000 7f59677fe6c0 1 -- 10.20.193.27:0/97063904 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f59680cd970 con 0x7f5980115440 2026-02-13T02:23:53.194 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.189+0000 7f597e5756c0 1 --2- 10.20.193.27:0/97063904 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f59580b5600 0x7f59580d5ab0 secure :-1 s=READY pgs=255 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5974005980 tx=0x7f5974005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:53.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.305+0000 7f5984fe86c0 1 -- 10.20.193.27:0/97063904 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f598010f600 con 0x7f5980115440 2026-02-13T02:23:53.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.305+0000 7f59677fe6c0 1 -- 10.20.193.27:0/97063904 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f59680cdb50 con 0x7f5980115440 2026-02-13T02:23:53.308 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:23:53.308 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:23:53.309 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.306+0000 7f5984fe86c0 1 -- 10.20.193.27:0/97063904 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f59580b5600 msgr2=0x7f59580d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:53.309 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.306+0000 7f5984fe86c0 1 --2- 10.20.193.27:0/97063904 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f59580b5600 0x7f59580d5ab0 secure :-1 s=READY pgs=255 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5974005980 tx=0x7f5974005910 comp rx=0 tx=0).stop 2026-02-13T02:23:53.309 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.306+0000 7f5984fe86c0 1 -- 10.20.193.27:0/97063904 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5980115440 msgr2=0x7f5980175ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:53.309 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.306+0000 7f5984fe86c0 1 --2- 10.20.193.27:0/97063904 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5980115440 0x7f5980175ce0 secure :-1 s=READY pgs=383 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5968004830 tx=0x7f596802fbf0 comp rx=0 tx=0).stop 2026-02-13T02:23:53.310 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.306+0000 7f5984fe86c0 1 -- 10.20.193.27:0/97063904 shutdown_connections 2026-02-13T02:23:53.310 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.306+0000 7f5984fe86c0 1 --2- 10.20.193.27:0/97063904 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f59580b5600 0x7f59580d5ab0 unknown :-1 s=CLOSED pgs=255 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:53.310 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.306+0000 7f5984fe86c0 1 --2- 10.20.193.27:0/97063904 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f59800f5290 0x7f59800fa560 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:53.310 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.306+0000 7f5984fe86c0 1 --2- 10.20.193.27:0/97063904 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5980115440 0x7f5980175ce0 unknown :-1 s=CLOSED pgs=383 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:53.310 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.306+0000 7f5984fe86c0 1 --2- 10.20.193.27:0/97063904 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f598010f230 0x7f59800f4c60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:53.310 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.306+0000 7f5984fe86c0 1 -- 10.20.193.27:0/97063904 >> 10.20.193.27:0/97063904 conn(0x7f59800ecb30 msgr2=0x7f598010d930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:53.310 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.306+0000 7f5984fe86c0 1 -- 10.20.193.27:0/97063904 shutdown_connections 2026-02-13T02:23:53.310 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:53.306+0000 7f5984fe86c0 1 -- 10.20.193.27:0/97063904 wait complete. 2026-02-13T02:23:54.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:54 trial028 ceph-mon[26701]: pgmap v235: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:23:54.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:54 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/97063904' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:54.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:54 trial107 ceph-mon[27422]: pgmap v235: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:23:54.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:54 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/97063904' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:54.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:54 trial027 ceph-mon[24567]: pgmap v235: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:23:54.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:54 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/97063904' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:56.349 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:23:56.471 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:23:56.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:56 trial028 ceph-mon[26701]: pgmap v236: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:23:56.494 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:56 trial027 ceph-mon[24567]: pgmap v236: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:23:56.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:56 trial107 ceph-mon[27422]: pgmap v236: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:23:56.589 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.584+0000 7f2e3aed86c0 1 -- 10.20.193.27:0/361041851 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2e3410cf80 msgr2=0x7f2e3410d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:56.589 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.584+0000 7f2e3aed86c0 1 --2- 10.20.193.27:0/361041851 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2e3410cf80 0x7f2e3410d350 secure :-1 s=READY pgs=384 gs=1 cs=0 l=1 c_cookie=9063458928865075625 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2e240099a0 tx=0x7f2e2402ed20 comp rx=0 tx=0).stop 2026-02-13T02:23:56.589 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.584+0000 7f2e3aed86c0 1 -- 10.20.193.27:0/361041851 shutdown_connections 2026-02-13T02:23:56.589 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.584+0000 7f2e3aed86c0 1 --2- 10.20.193.27:0/361041851 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2e34113190 0x7f2e34173b30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:56.589 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.584+0000 7f2e3aed86c0 1 --2- 10.20.193.27:0/361041851 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2e3410d970 0x7f2e341129b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:56.589 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.584+0000 7f2e3aed86c0 1 --2- 10.20.193.27:0/361041851 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2e3410cf80 0x7f2e3410d350 unknown :-1 s=CLOSED pgs=384 gs=1 cs=0 l=1 c_cookie=9063458928865075625 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:56.589 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.584+0000 7f2e3aed86c0 1 -- 10.20.193.27:0/361041851 >> 10.20.193.27:0/361041851 conn(0x7f2e340ea880 msgr2=0x7f2e3410ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:56.590 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.584+0000 7f2e3aed86c0 1 -- 10.20.193.27:0/361041851 shutdown_connections 2026-02-13T02:23:56.590 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.584+0000 7f2e3aed86c0 1 -- 10.20.193.27:0/361041851 wait complete. 2026-02-13T02:23:56.590 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.585+0000 7f2e3aed86c0 1 Processor -- start 2026-02-13T02:23:56.590 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.585+0000 7f2e3aed86c0 1 -- start start 2026-02-13T02:23:56.590 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.585+0000 7f2e3aed86c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2e3410cf80 0x7f2e34062070 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:56.590 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.585+0000 7f2e3aed86c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2e34113190 0x7f2e340605e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:56.590 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.585+0000 7f2e3aed86c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2e34060b20 0x7f2e34060fc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:56.590 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.585+0000 7f2e3aed86c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2e34167d80 con 0x7f2e34113190 2026-02-13T02:23:56.590 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.585+0000 7f2e3aed86c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2e34167ef0 con 0x7f2e34060b20 2026-02-13T02:23:56.590 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.585+0000 7f2e3aed86c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2e341680b0 con 0x7f2e3410cf80 2026-02-13T02:23:56.590 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.585+0000 7f2e2bfff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2e34113190 0x7f2e340605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:56.590 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.585+0000 7f2e2bfff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2e34113190 0x7f2e340605e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:34112/0 (socket says 10.20.193.27:34112) 2026-02-13T02:23:56.591 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.585+0000 7f2e2bfff6c0 1 -- 10.20.193.27:0/1228944060 learned_addr learned my addr 10.20.193.27:0/1228944060 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:23:56.591 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.585+0000 7f2e38c4d6c0 1 --2- 10.20.193.27:0/1228944060 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2e3410cf80 0x7f2e34062070 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:56.591 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.585+0000 7f2e2bfff6c0 1 -- 10.20.193.27:0/1228944060 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2e3410cf80 msgr2=0x7f2e34062070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:56.591 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.585+0000 7f2e2bfff6c0 1 --2- 10.20.193.27:0/1228944060 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2e3410cf80 0x7f2e34062070 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:56.591 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.585+0000 7f2e2bfff6c0 1 -- 10.20.193.27:0/1228944060 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2e34060b20 msgr2=0x7f2e34060fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:56.591 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.585+0000 7f2e2bfff6c0 1 --2- 10.20.193.27:0/1228944060 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2e34060b20 0x7f2e34060fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:56.591 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.585+0000 7f2e2bfff6c0 1 -- 10.20.193.27:0/1228944060 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e24009590 con 0x7f2e34113190 2026-02-13T02:23:56.591 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.585+0000 7f2e2bfff6c0 1 --2- 10.20.193.27:0/1228944060 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2e34113190 0x7f2e340605e0 secure :-1 s=READY pgs=385 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2e1c00b6c0 tx=0x7f2e1c00bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:56.591 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.585+0000 7f2e29ffb6c0 1 -- 10.20.193.27:0/1228944060 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2e1c004250 con 0x7f2e34113190 2026-02-13T02:23:56.591 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.585+0000 7f2e29ffb6c0 1 -- 10.20.193.27:0/1228944060 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f2e1c009070 con 0x7f2e34113190 2026-02-13T02:23:56.591 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.585+0000 7f2e3aed86c0 1 -- 10.20.193.27:0/1228944060 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e34168390 con 0x7f2e34113190 2026-02-13T02:23:56.591 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.585+0000 7f2e29ffb6c0 1 -- 10.20.193.27:0/1228944060 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2e1c00ccb0 con 0x7f2e34113190 2026-02-13T02:23:56.592 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.585+0000 7f2e3aed86c0 1 -- 10.20.193.27:0/1228944060 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e34168870 con 0x7f2e34113190 2026-02-13T02:23:56.592 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.586+0000 7f2e3aed86c0 1 -- 10.20.193.27:0/1228944060 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e3410e250 con 0x7f2e34113190 2026-02-13T02:23:56.593 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.587+0000 7f2e29ffb6c0 1 -- 10.20.193.27:0/1228944060 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f2e1c0028c0 con 0x7f2e34113190 2026-02-13T02:23:56.593 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.588+0000 7f2e29ffb6c0 1 --2- 10.20.193.27:0/1228944060 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f2e100b5680 0x7f2e100d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:23:56.593 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.588+0000 7f2e38c4d6c0 1 --2- 10.20.193.27:0/1228944060 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f2e100b5680 0x7f2e100d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:23:56.593 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.588+0000 7f2e29ffb6c0 1 -- 10.20.193.27:0/1228944060 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f2e1c0e3a60 con 0x7f2e34113190 2026-02-13T02:23:56.593 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.588+0000 7f2e38c4d6c0 1 --2- 10.20.193.27:0/1228944060 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f2e100b5680 0x7f2e100d5b30 secure :-1 s=READY pgs=256 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2e240098c0 tx=0x7f2e24005f40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:23:56.593 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.588+0000 7f2e29ffb6c0 1 -- 10.20.193.27:0/1228944060 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f2e1c0a64e0 con 0x7f2e34113190 2026-02-13T02:23:56.707 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.705+0000 7f2e3aed86c0 1 -- 10.20.193.27:0/1228944060 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2e3410d350 con 0x7f2e34113190 2026-02-13T02:23:56.708 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.705+0000 7f2e29ffb6c0 1 -- 10.20.193.27:0/1228944060 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f2e1c0a9690 con 0x7f2e34113190 2026-02-13T02:23:56.708 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:23:56.708 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:23:56.709 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.706+0000 7f2e3aed86c0 1 -- 10.20.193.27:0/1228944060 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f2e100b5680 msgr2=0x7f2e100d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:56.709 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.706+0000 7f2e3aed86c0 1 --2- 10.20.193.27:0/1228944060 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f2e100b5680 0x7f2e100d5b30 secure :-1 s=READY pgs=256 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2e240098c0 tx=0x7f2e24005f40 comp rx=0 tx=0).stop 2026-02-13T02:23:56.709 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.706+0000 7f2e3aed86c0 1 -- 10.20.193.27:0/1228944060 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2e34113190 msgr2=0x7f2e340605e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:23:56.709 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.706+0000 7f2e3aed86c0 1 --2- 10.20.193.27:0/1228944060 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2e34113190 0x7f2e340605e0 secure :-1 s=READY pgs=385 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2e1c00b6c0 tx=0x7f2e1c00bb90 comp rx=0 tx=0).stop 2026-02-13T02:23:56.709 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.707+0000 7f2e3aed86c0 1 -- 10.20.193.27:0/1228944060 shutdown_connections 2026-02-13T02:23:56.710 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.707+0000 7f2e3aed86c0 1 --2- 10.20.193.27:0/1228944060 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f2e100b5680 0x7f2e100d5b30 unknown :-1 s=CLOSED pgs=256 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:56.710 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.707+0000 7f2e3aed86c0 1 --2- 10.20.193.27:0/1228944060 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2e34060b20 0x7f2e34060fc0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:56.710 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.707+0000 7f2e3aed86c0 1 --2- 10.20.193.27:0/1228944060 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2e34113190 0x7f2e340605e0 unknown :-1 s=CLOSED pgs=385 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:56.710 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.707+0000 7f2e3aed86c0 1 --2- 10.20.193.27:0/1228944060 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2e3410cf80 0x7f2e34062070 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:23:56.710 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.707+0000 7f2e3aed86c0 1 -- 10.20.193.27:0/1228944060 >> 10.20.193.27:0/1228944060 conn(0x7f2e340ea880 msgr2=0x7f2e3410ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:23:56.710 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.707+0000 7f2e3aed86c0 1 -- 10.20.193.27:0/1228944060 shutdown_connections 2026-02-13T02:23:56.710 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:23:56.707+0000 7f2e3aed86c0 1 -- 10.20.193.27:0/1228944060 wait complete. 2026-02-13T02:23:57.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:57 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1228944060' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:57.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:57 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1228944060' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:57.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:57 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1228944060' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:23:58.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:23:58 trial028 ceph-mon[26701]: pgmap v237: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:23:58.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:23:58 trial107 ceph-mon[27422]: pgmap v237: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:23:58.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:23:58 trial027 ceph-mon[24567]: pgmap v237: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:23:59.772 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:23:59.889 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:24:00.010 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.007+0000 7fc587a886c0 1 -- 10.20.193.27:0/2104955881 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc580114190 msgr2=0x7fc580119140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:00.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.007+0000 7fc587a886c0 1 --2- 10.20.193.27:0/2104955881 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc580114190 0x7fc580119140 secure :-1 s=READY pgs=386 gs=1 cs=0 l=1 c_cookie=14966698421498204793 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc574009970 tx=0x7fc57402ed50 comp rx=0 tx=0).stop 2026-02-13T02:24:00.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.007+0000 7fc587a886c0 1 -- 10.20.193.27:0/2104955881 shutdown_connections 2026-02-13T02:24:00.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.007+0000 7fc587a886c0 1 --2- 10.20.193.27:0/2104955881 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc5801198f0 0x7fc58017a2c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:00.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.007+0000 7fc587a886c0 1 --2- 10.20.193.27:0/2104955881 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc580114190 0x7fc580119140 unknown :-1 s=CLOSED pgs=386 gs=1 cs=0 l=1 c_cookie=14966698421498204793 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:00.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.007+0000 7fc587a886c0 1 --2- 10.20.193.27:0/2104955881 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc5801137a0 0x7fc580113b70 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:00.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.007+0000 7fc587a886c0 1 -- 10.20.193.27:0/2104955881 >> 10.20.193.27:0/2104955881 conn(0x7fc580060760 msgr2=0x7fc580060d00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:00.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.007+0000 7fc587a886c0 1 -- 10.20.193.27:0/2104955881 shutdown_connections 2026-02-13T02:24:00.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.007+0000 7fc587a886c0 1 -- 10.20.193.27:0/2104955881 wait complete. 2026-02-13T02:24:00.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.007+0000 7fc587a886c0 1 Processor -- start 2026-02-13T02:24:00.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.007+0000 7fc587a886c0 1 -- start start 2026-02-13T02:24:00.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.008+0000 7fc587a886c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc5801137a0 0x7fc580121800 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:00.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.008+0000 7fc587a886c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc5801198f0 0x7fc580121d40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:00.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.008+0000 7fc587a886c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc5801270b0 0x7fc580127550 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:00.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.008+0000 7fc587a886c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc580127a90 con 0x7fc5801198f0 2026-02-13T02:24:00.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.008+0000 7fc587a886c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc580127c00 con 0x7fc5801270b0 2026-02-13T02:24:00.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.008+0000 7fc587a886c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc580127dc0 con 0x7fc5801137a0 2026-02-13T02:24:00.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.008+0000 7fc584ff86c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc5801198f0 0x7fc580121d40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:00.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.008+0000 7fc584ff86c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc5801198f0 0x7fc580121d40 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:60280/0 (socket says 10.20.193.27:60280) 2026-02-13T02:24:00.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.008+0000 7fc584ff86c0 1 -- 10.20.193.27:0/2867438694 learned_addr learned my addr 10.20.193.27:0/2867438694 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:24:00.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.008+0000 7fc5857f96c0 1 --2- 10.20.193.27:0/2867438694 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc5801137a0 0x7fc580121800 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:00.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.008+0000 7fc585ffa6c0 1 --2- 10.20.193.27:0/2867438694 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc5801270b0 0x7fc580127550 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:00.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.008+0000 7fc584ff86c0 1 -- 10.20.193.27:0/2867438694 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc5801137a0 msgr2=0x7fc580121800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:00.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.008+0000 7fc584ff86c0 1 --2- 10.20.193.27:0/2867438694 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc5801137a0 0x7fc580121800 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:00.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.008+0000 7fc584ff86c0 1 -- 10.20.193.27:0/2867438694 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc5801270b0 msgr2=0x7fc580127550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:00.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.008+0000 7fc584ff86c0 1 --2- 10.20.193.27:0/2867438694 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc5801270b0 0x7fc580127550 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:00.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.008+0000 7fc584ff86c0 1 -- 10.20.193.27:0/2867438694 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc574009590 con 0x7fc5801198f0 2026-02-13T02:24:00.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.008+0000 7fc584ff86c0 1 --2- 10.20.193.27:0/2867438694 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc5801198f0 0x7fc580121d40 secure :-1 s=READY pgs=387 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc5740092e0 tx=0x7fc57402fbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:00.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.008+0000 7fc56e7fc6c0 1 -- 10.20.193.27:0/2867438694 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc574002ae0 con 0x7fc5801198f0 2026-02-13T02:24:00.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.008+0000 7fc56e7fc6c0 1 -- 10.20.193.27:0/2867438694 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc574002c40 con 0x7fc5801198f0 2026-02-13T02:24:00.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.008+0000 7fc587a886c0 1 -- 10.20.193.27:0/2867438694 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc580168240 con 0x7fc5801198f0 2026-02-13T02:24:00.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.008+0000 7fc56e7fc6c0 1 -- 10.20.193.27:0/2867438694 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc5740372f0 con 0x7fc5801198f0 2026-02-13T02:24:00.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.008+0000 7fc587a886c0 1 -- 10.20.193.27:0/2867438694 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5801686c0 con 0x7fc5801198f0 2026-02-13T02:24:00.013 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.008+0000 7fc587a886c0 1 -- 10.20.193.27:0/2867438694 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5801149e0 con 0x7fc5801198f0 2026-02-13T02:24:00.015 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.010+0000 7fc56e7fc6c0 1 -- 10.20.193.27:0/2867438694 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7fc5740077a0 con 0x7fc5801198f0 2026-02-13T02:24:00.015 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.010+0000 7fc56e7fc6c0 1 --2- 10.20.193.27:0/2867438694 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc55c0b5600 0x7fc55c0d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:00.015 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.010+0000 7fc5857f96c0 1 --2- 10.20.193.27:0/2867438694 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc55c0b5600 0x7fc55c0d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:00.015 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.011+0000 7fc56e7fc6c0 1 -- 10.20.193.27:0/2867438694 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7fc574107d80 con 0x7fc5801198f0 2026-02-13T02:24:00.015 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.011+0000 7fc5857f96c0 1 --2- 10.20.193.27:0/2867438694 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc55c0b5600 0x7fc55c0d5ab0 secure :-1 s=READY pgs=257 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc570004750 tx=0x7fc570009260 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:00.015 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.011+0000 7fc56e7fc6c0 1 -- 10.20.193.27:0/2867438694 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fc5740ca800 con 0x7fc5801198f0 2026-02-13T02:24:00.127 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.126+0000 7fc587a886c0 1 -- 10.20.193.27:0/2867438694 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc580113b70 con 0x7fc5801198f0 2026-02-13T02:24:00.127 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.126+0000 7fc56e7fc6c0 1 -- 10.20.193.27:0/2867438694 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7fc5740cd9b0 con 0x7fc5801198f0 2026-02-13T02:24:00.128 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:24:00.128 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:24:00.129 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.127+0000 7fc587a886c0 1 -- 10.20.193.27:0/2867438694 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc55c0b5600 msgr2=0x7fc55c0d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:00.129 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.127+0000 7fc587a886c0 1 --2- 10.20.193.27:0/2867438694 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc55c0b5600 0x7fc55c0d5ab0 secure :-1 s=READY pgs=257 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc570004750 tx=0x7fc570009260 comp rx=0 tx=0).stop 2026-02-13T02:24:00.129 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.127+0000 7fc587a886c0 1 -- 10.20.193.27:0/2867438694 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc5801198f0 msgr2=0x7fc580121d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:00.129 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.127+0000 7fc587a886c0 1 --2- 10.20.193.27:0/2867438694 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc5801198f0 0x7fc580121d40 secure :-1 s=READY pgs=387 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc5740092e0 tx=0x7fc57402fbf0 comp rx=0 tx=0).stop 2026-02-13T02:24:00.129 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.127+0000 7fc587a886c0 1 -- 10.20.193.27:0/2867438694 shutdown_connections 2026-02-13T02:24:00.129 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.127+0000 7fc587a886c0 1 --2- 10.20.193.27:0/2867438694 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc55c0b5600 0x7fc55c0d5ab0 unknown :-1 s=CLOSED pgs=257 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:00.129 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.127+0000 7fc587a886c0 1 --2- 10.20.193.27:0/2867438694 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc5801270b0 0x7fc580127550 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:00.129 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.127+0000 7fc587a886c0 1 --2- 10.20.193.27:0/2867438694 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc5801198f0 0x7fc580121d40 unknown :-1 s=CLOSED pgs=387 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:00.129 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.127+0000 7fc587a886c0 1 --2- 10.20.193.27:0/2867438694 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc5801137a0 0x7fc580121800 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:00.130 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.127+0000 7fc587a886c0 1 -- 10.20.193.27:0/2867438694 >> 10.20.193.27:0/2867438694 conn(0x7fc580060760 msgr2=0x7fc580117fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:00.130 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.127+0000 7fc587a886c0 1 -- 10.20.193.27:0/2867438694 shutdown_connections 2026-02-13T02:24:00.130 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:00.127+0000 7fc587a886c0 1 -- 10.20.193.27:0/2867438694 wait complete. 2026-02-13T02:24:00.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:00 trial027 ceph-mon[24567]: pgmap v238: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:00.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:00 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2867438694' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:00.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:00 trial028 ceph-mon[26701]: pgmap v238: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:00.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:00 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2867438694' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:00.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:00 trial107 ceph-mon[27422]: pgmap v238: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:00.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:00 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2867438694' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:02.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:02 trial028 ceph-mon[26701]: pgmap v239: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:02.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:02 trial107 ceph-mon[27422]: pgmap v239: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:02.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:02 trial027 ceph-mon[24567]: pgmap v239: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:03.170 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:24:03.289 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:24:03.405 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.402+0000 7f6e92ddb6c0 1 -- 10.20.193.27:0/2873802937 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f6e8c115410 msgr2=0x7f6e8c175db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:03.405 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.402+0000 7f6e92ddb6c0 1 --2- 10.20.193.27:0/2873802937 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f6e8c115410 0x7f6e8c175db0 secure :-1 s=READY pgs=388 gs=1 cs=0 l=1 c_cookie=1582075929802107694 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6e8800aa70 tx=0x7f6e8802fde0 comp rx=0 tx=0).stop 2026-02-13T02:24:03.405 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.402+0000 7f6e92ddb6c0 1 -- 10.20.193.27:0/2873802937 shutdown_connections 2026-02-13T02:24:03.405 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.402+0000 7f6e92ddb6c0 1 --2- 10.20.193.27:0/2873802937 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f6e8c115410 0x7f6e8c175db0 unknown :-1 s=CLOSED pgs=388 gs=1 cs=0 l=1 c_cookie=1582075929802107694 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:03.405 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.402+0000 7f6e92ddb6c0 1 --2- 10.20.193.27:0/2873802937 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f6e8c10fbf0 0x7f6e8c114c30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:03.406 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.402+0000 7f6e92ddb6c0 1 --2- 10.20.193.27:0/2873802937 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6e8c10f200 0x7f6e8c10f5d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:03.406 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.402+0000 7f6e92ddb6c0 1 -- 10.20.193.27:0/2873802937 >> 10.20.193.27:0/2873802937 conn(0x7f6e8c0ecb00 msgr2=0x7f6e8c10cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:03.406 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.402+0000 7f6e92ddb6c0 1 -- 10.20.193.27:0/2873802937 shutdown_connections 2026-02-13T02:24:03.406 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.402+0000 7f6e92ddb6c0 1 -- 10.20.193.27:0/2873802937 wait complete. 2026-02-13T02:24:03.406 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.402+0000 7f6e92ddb6c0 1 Processor -- start 2026-02-13T02:24:03.406 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.402+0000 7f6e92ddb6c0 1 -- start start 2026-02-13T02:24:03.406 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.402+0000 7f6e92ddb6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f6e8c10f200 0x7f6e8c11d250 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:03.406 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.402+0000 7f6e92ddb6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6e8c115410 0x7f6e8c11d790 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:03.406 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.402+0000 7f6e92ddb6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f6e8c122b50 0x7f6e8c122ff0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:03.406 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.402+0000 7f6e92ddb6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8c123560 con 0x7f6e8c10f200 2026-02-13T02:24:03.407 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.402+0000 7f6e92ddb6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8c1236d0 con 0x7f6e8c115410 2026-02-13T02:24:03.407 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.402+0000 7f6e92ddb6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8c123890 con 0x7f6e8c122b50 2026-02-13T02:24:03.407 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.402+0000 7f6e90b4c6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f6e8c10f200 0x7f6e8c11d250 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:03.407 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.402+0000 7f6e90b4c6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f6e8c10f200 0x7f6e8c11d250 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:60298/0 (socket says 10.20.193.27:60298) 2026-02-13T02:24:03.407 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.402+0000 7f6e90b4c6c0 1 -- 10.20.193.27:0/1820380068 learned_addr learned my addr 10.20.193.27:0/1820380068 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:24:03.407 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.402+0000 7f6e83fff6c0 1 --2- 10.20.193.27:0/1820380068 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6e8c115410 0x7f6e8c11d790 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:03.407 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.402+0000 7f6e90b4c6c0 1 -- 10.20.193.27:0/1820380068 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f6e8c122b50 msgr2=0x7f6e8c122ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:03.407 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.402+0000 7f6e90b4c6c0 1 --2- 10.20.193.27:0/1820380068 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f6e8c122b50 0x7f6e8c122ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:03.407 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.403+0000 7f6e90b4c6c0 1 -- 10.20.193.27:0/1820380068 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6e8c115410 msgr2=0x7f6e8c11d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:03.407 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.403+0000 7f6e90b4c6c0 1 --2- 10.20.193.27:0/1820380068 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6e8c115410 0x7f6e8c11d790 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:03.407 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.403+0000 7f6e90b4c6c0 1 -- 10.20.193.27:0/1820380068 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e8800a660 con 0x7f6e8c10f200 2026-02-13T02:24:03.407 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.403+0000 7f6e90b4c6c0 1 --2- 10.20.193.27:0/1820380068 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f6e8c10f200 0x7f6e8c11d250 secure :-1 s=READY pgs=389 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6e7c00b710 tx=0x7f6e7c00bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:03.407 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.403+0000 7f6e81ffb6c0 1 -- 10.20.193.27:0/1820380068 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6e7c004250 con 0x7f6e8c10f200 2026-02-13T02:24:03.408 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.403+0000 7f6e92ddb6c0 1 -- 10.20.193.27:0/1820380068 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e8c163d40 con 0x7f6e8c10f200 2026-02-13T02:24:03.408 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.403+0000 7f6e81ffb6c0 1 -- 10.20.193.27:0/1820380068 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6e7c002d80 con 0x7f6e8c10f200 2026-02-13T02:24:03.408 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.403+0000 7f6e81ffb6c0 1 -- 10.20.193.27:0/1820380068 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6e7c00cdc0 con 0x7f6e8c10f200 2026-02-13T02:24:03.408 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.403+0000 7f6e92ddb6c0 1 -- 10.20.193.27:0/1820380068 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e8c1641d0 con 0x7f6e8c10f200 2026-02-13T02:24:03.408 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.403+0000 7f6e92ddb6c0 1 -- 10.20.193.27:0/1820380068 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e8c1104d0 con 0x7f6e8c10f200 2026-02-13T02:24:03.409 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.405+0000 7f6e81ffb6c0 1 -- 10.20.193.27:0/1820380068 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f6e7c0028c0 con 0x7f6e8c10f200 2026-02-13T02:24:03.409 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.405+0000 7f6e81ffb6c0 1 --2- 10.20.193.27:0/1820380068 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f6e680b5600 0x7f6e680d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:03.409 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.406+0000 7f6e83fff6c0 1 --2- 10.20.193.27:0/1820380068 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f6e680b5600 0x7f6e680d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:03.409 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.406+0000 7f6e81ffb6c0 1 -- 10.20.193.27:0/1820380068 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f6e7c0e3bf0 con 0x7f6e8c10f200 2026-02-13T02:24:03.409 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.406+0000 7f6e81ffb6c0 1 -- 10.20.193.27:0/1820380068 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f6e7c0a9860 con 0x7f6e8c10f200 2026-02-13T02:24:03.410 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.406+0000 7f6e83fff6c0 1 --2- 10.20.193.27:0/1820380068 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f6e680b5600 0x7f6e680d5ab0 secure :-1 s=READY pgs=258 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6e8c123be0 tx=0x7f6e74009590 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:03.520 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.519+0000 7f6e92ddb6c0 1 -- 10.20.193.27:0/1820380068 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6e8c10f5d0 con 0x7f6e8c10f200 2026-02-13T02:24:03.520 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.519+0000 7f6e81ffb6c0 1 -- 10.20.193.27:0/1820380068 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f6e7c0a9a40 con 0x7f6e8c10f200 2026-02-13T02:24:03.521 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:24:03.521 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:24:03.522 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.520+0000 7f6e92ddb6c0 1 -- 10.20.193.27:0/1820380068 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f6e680b5600 msgr2=0x7f6e680d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:03.522 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.520+0000 7f6e92ddb6c0 1 --2- 10.20.193.27:0/1820380068 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f6e680b5600 0x7f6e680d5ab0 secure :-1 s=READY pgs=258 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6e8c123be0 tx=0x7f6e74009590 comp rx=0 tx=0).stop 2026-02-13T02:24:03.522 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.520+0000 7f6e92ddb6c0 1 -- 10.20.193.27:0/1820380068 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f6e8c10f200 msgr2=0x7f6e8c11d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:03.522 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.520+0000 7f6e92ddb6c0 1 --2- 10.20.193.27:0/1820380068 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f6e8c10f200 0x7f6e8c11d250 secure :-1 s=READY pgs=389 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6e7c00b710 tx=0x7f6e7c00bbe0 comp rx=0 tx=0).stop 2026-02-13T02:24:03.522 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.520+0000 7f6e92ddb6c0 1 -- 10.20.193.27:0/1820380068 shutdown_connections 2026-02-13T02:24:03.522 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.520+0000 7f6e92ddb6c0 1 --2- 10.20.193.27:0/1820380068 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f6e680b5600 0x7f6e680d5ab0 unknown :-1 s=CLOSED pgs=258 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:03.522 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.520+0000 7f6e92ddb6c0 1 --2- 10.20.193.27:0/1820380068 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f6e8c122b50 0x7f6e8c122ff0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:03.522 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.520+0000 7f6e92ddb6c0 1 --2- 10.20.193.27:0/1820380068 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6e8c115410 0x7f6e8c11d790 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:03.522 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.520+0000 7f6e92ddb6c0 1 --2- 10.20.193.27:0/1820380068 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f6e8c10f200 0x7f6e8c11d250 unknown :-1 s=CLOSED pgs=389 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:03.523 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.520+0000 7f6e92ddb6c0 1 -- 10.20.193.27:0/1820380068 >> 10.20.193.27:0/1820380068 conn(0x7f6e8c0ecb00 msgr2=0x7f6e8c10cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:03.523 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.520+0000 7f6e92ddb6c0 1 -- 10.20.193.27:0/1820380068 shutdown_connections 2026-02-13T02:24:03.523 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:03.520+0000 7f6e92ddb6c0 1 -- 10.20.193.27:0/1820380068 wait complete. 2026-02-13T02:24:04.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:04 trial028 ceph-mon[26701]: pgmap v240: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:24:04.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:04 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1820380068' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:04.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:04 trial107 ceph-mon[27422]: pgmap v240: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:24:04.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:04 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1820380068' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:04.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:04 trial027 ceph-mon[24567]: pgmap v240: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:24:04.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:04 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1820380068' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:06.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:06 trial028 ceph-mon[26701]: pgmap v241: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:06.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:06 trial107 ceph-mon[27422]: pgmap v241: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:06.562 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:24:06.587 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:06 trial027 ceph-mon[24567]: pgmap v241: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:06.680 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:24:06.805 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.801+0000 7f16378786c0 1 -- 10.20.193.27:0/1510953925 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f163010f230 msgr2=0x7f163010f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:06.805 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.801+0000 7f16378786c0 1 --2- 10.20.193.27:0/1510953925 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f163010f230 0x7f163010f600 secure :-1 s=READY pgs=390 gs=1 cs=0 l=1 c_cookie=6478599038111372754 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f16200099a0 tx=0x7f162002ed20 comp rx=0 tx=0).stop 2026-02-13T02:24:06.805 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.801+0000 7f16378786c0 1 -- 10.20.193.27:0/1510953925 shutdown_connections 2026-02-13T02:24:06.805 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.801+0000 7f16378786c0 1 --2- 10.20.193.27:0/1510953925 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1630115440 0x7f1630175de0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:06.805 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.801+0000 7f16378786c0 1 --2- 10.20.193.27:0/1510953925 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f163010fc20 0x7f1630114c60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:06.805 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.801+0000 7f16378786c0 1 --2- 10.20.193.27:0/1510953925 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f163010f230 0x7f163010f600 unknown :-1 s=CLOSED pgs=390 gs=1 cs=0 l=1 c_cookie=6478599038111372754 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:06.805 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.801+0000 7f16378786c0 1 -- 10.20.193.27:0/1510953925 >> 10.20.193.27:0/1510953925 conn(0x7f16300ecb30 msgr2=0x7f163010cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:06.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.801+0000 7f16378786c0 1 -- 10.20.193.27:0/1510953925 shutdown_connections 2026-02-13T02:24:06.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.801+0000 7f16378786c0 1 -- 10.20.193.27:0/1510953925 wait complete. 2026-02-13T02:24:06.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.802+0000 7f16378786c0 1 Processor -- start 2026-02-13T02:24:06.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.802+0000 7f16378786c0 1 -- start start 2026-02-13T02:24:06.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.802+0000 7f16378786c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f163010f230 0x7f163011c8e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:06.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.802+0000 7f16378786c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1630115440 0x7f163011ce20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:06.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.802+0000 7f16378786c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f16301221e0 0x7f1630122680 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:06.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.802+0000 7f16378786c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1630122bf0 con 0x7f163010f230 2026-02-13T02:24:06.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.802+0000 7f16378786c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1630122d60 con 0x7f1630115440 2026-02-13T02:24:06.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.802+0000 7f16378786c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1630122f20 con 0x7f16301221e0 2026-02-13T02:24:06.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.802+0000 7f16355ed6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f163010f230 0x7f163011c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:06.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.802+0000 7f16355ed6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f163010f230 0x7f163011c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:60318/0 (socket says 10.20.193.27:60318) 2026-02-13T02:24:06.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.802+0000 7f16355ed6c0 1 -- 10.20.193.27:0/69230182 learned_addr learned my addr 10.20.193.27:0/69230182 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:24:06.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.802+0000 7f1635dee6c0 1 --2- 10.20.193.27:0/69230182 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f16301221e0 0x7f1630122680 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:06.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.802+0000 7f1634dec6c0 1 --2- 10.20.193.27:0/69230182 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1630115440 0x7f163011ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:06.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.802+0000 7f16355ed6c0 1 -- 10.20.193.27:0/69230182 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f16301221e0 msgr2=0x7f1630122680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:06.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.802+0000 7f16355ed6c0 1 --2- 10.20.193.27:0/69230182 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f16301221e0 0x7f1630122680 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:06.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.802+0000 7f16355ed6c0 1 -- 10.20.193.27:0/69230182 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1630115440 msgr2=0x7f163011ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:06.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.802+0000 7f16355ed6c0 1 --2- 10.20.193.27:0/69230182 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1630115440 0x7f163011ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:06.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.802+0000 7f16355ed6c0 1 -- 10.20.193.27:0/69230182 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1620009590 con 0x7f163010f230 2026-02-13T02:24:06.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.802+0000 7f16355ed6c0 1 --2- 10.20.193.27:0/69230182 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f163010f230 0x7f163011c8e0 secure :-1 s=READY pgs=391 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f162002f230 tx=0x7f162002fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:06.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.802+0000 7f161e7fc6c0 1 -- 10.20.193.27:0/69230182 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1620002ba0 con 0x7f163010f230 2026-02-13T02:24:06.807 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.802+0000 7f161e7fc6c0 1 -- 10.20.193.27:0/69230182 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1620002d00 con 0x7f163010f230 2026-02-13T02:24:06.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.802+0000 7f16378786c0 1 -- 10.20.193.27:0/69230182 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1630163370 con 0x7f163010f230 2026-02-13T02:24:06.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.802+0000 7f161e7fc6c0 1 -- 10.20.193.27:0/69230182 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f16200372f0 con 0x7f163010f230 2026-02-13T02:24:06.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.802+0000 7f16378786c0 1 -- 10.20.193.27:0/69230182 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16301637a0 con 0x7f163010f230 2026-02-13T02:24:06.808 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.803+0000 7f16378786c0 1 -- 10.20.193.27:0/69230182 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1630110500 con 0x7f163010f230 2026-02-13T02:24:06.809 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.804+0000 7f161e7fc6c0 1 -- 10.20.193.27:0/69230182 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f16200078d0 con 0x7f163010f230 2026-02-13T02:24:06.809 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.805+0000 7f161e7fc6c0 1 --2- 10.20.193.27:0/69230182 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f160c0b5600 0x7f160c0d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:06.809 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.805+0000 7f1634dec6c0 1 --2- 10.20.193.27:0/69230182 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f160c0b5600 0x7f160c0d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:06.809 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.805+0000 7f161e7fc6c0 1 -- 10.20.193.27:0/69230182 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f1620107c30 con 0x7f163010f230 2026-02-13T02:24:06.809 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.805+0000 7f1634dec6c0 1 --2- 10.20.193.27:0/69230182 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f160c0b5600 0x7f160c0d5ab0 secure :-1 s=READY pgs=259 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1624005980 tx=0x7f1624005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:06.810 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.805+0000 7f161e7fc6c0 1 -- 10.20.193.27:0/69230182 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f16200ca6b0 con 0x7f163010f230 2026-02-13T02:24:06.922 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.920+0000 7f16378786c0 1 -- 10.20.193.27:0/69230182 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f163010f600 con 0x7f163010f230 2026-02-13T02:24:06.922 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.920+0000 7f161e7fc6c0 1 -- 10.20.193.27:0/69230182 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f16200cd860 con 0x7f163010f230 2026-02-13T02:24:06.922 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:24:06.922 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:24:06.923 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.921+0000 7f16378786c0 1 -- 10.20.193.27:0/69230182 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f160c0b5600 msgr2=0x7f160c0d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:06.924 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.921+0000 7f16378786c0 1 --2- 10.20.193.27:0/69230182 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f160c0b5600 0x7f160c0d5ab0 secure :-1 s=READY pgs=259 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1624005980 tx=0x7f1624005910 comp rx=0 tx=0).stop 2026-02-13T02:24:06.924 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.921+0000 7f16378786c0 1 -- 10.20.193.27:0/69230182 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f163010f230 msgr2=0x7f163011c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:06.924 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.921+0000 7f16378786c0 1 --2- 10.20.193.27:0/69230182 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f163010f230 0x7f163011c8e0 secure :-1 s=READY pgs=391 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f162002f230 tx=0x7f162002fd40 comp rx=0 tx=0).stop 2026-02-13T02:24:06.924 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.921+0000 7f16378786c0 1 -- 10.20.193.27:0/69230182 shutdown_connections 2026-02-13T02:24:06.924 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.921+0000 7f16378786c0 1 --2- 10.20.193.27:0/69230182 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f160c0b5600 0x7f160c0d5ab0 unknown :-1 s=CLOSED pgs=259 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:06.924 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.921+0000 7f16378786c0 1 --2- 10.20.193.27:0/69230182 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f16301221e0 0x7f1630122680 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:06.924 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.921+0000 7f16378786c0 1 --2- 10.20.193.27:0/69230182 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1630115440 0x7f163011ce20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:06.924 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.921+0000 7f16378786c0 1 --2- 10.20.193.27:0/69230182 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f163010f230 0x7f163011c8e0 unknown :-1 s=CLOSED pgs=391 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:06.924 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.921+0000 7f16378786c0 1 -- 10.20.193.27:0/69230182 >> 10.20.193.27:0/69230182 conn(0x7f16300ecb30 msgr2=0x7f163010cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:06.924 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.921+0000 7f16378786c0 1 -- 10.20.193.27:0/69230182 shutdown_connections 2026-02-13T02:24:06.924 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:06.921+0000 7f16378786c0 1 -- 10.20.193.27:0/69230182 wait complete. 2026-02-13T02:24:07.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:07 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/69230182' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:07.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:07 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/69230182' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:07.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:07 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/69230182' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:08.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:08 trial028 ceph-mon[26701]: pgmap v242: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:24:08.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:08 trial107 ceph-mon[27422]: pgmap v242: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:24:08.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:08 trial027 ceph-mon[24567]: pgmap v242: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:24:09.964 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:24:10.081 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:24:10.198 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.196+0000 7f16e0b146c0 1 -- 10.20.193.27:0/1112766726 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f16dc10f200 msgr2=0x7f16dc10f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:10.198 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.196+0000 7f16e0b146c0 1 --2- 10.20.193.27:0/1112766726 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f16dc10f200 0x7f16dc10f5d0 secure :-1 s=READY pgs=392 gs=1 cs=0 l=1 c_cookie=15626150481215327687 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f16d00099a0 tx=0x7f16d002ed20 comp rx=0 tx=0).stop 2026-02-13T02:24:10.198 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.196+0000 7f16e0b146c0 1 -- 10.20.193.27:0/1112766726 shutdown_connections 2026-02-13T02:24:10.198 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.196+0000 7f16e0b146c0 1 --2- 10.20.193.27:0/1112766726 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f16dc115410 0x7f16dc175db0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:10.198 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.196+0000 7f16e0b146c0 1 --2- 10.20.193.27:0/1112766726 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f16dc10fbf0 0x7f16dc114c30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:10.198 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.196+0000 7f16e0b146c0 1 --2- 10.20.193.27:0/1112766726 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f16dc10f200 0x7f16dc10f5d0 unknown :-1 s=CLOSED pgs=392 gs=1 cs=0 l=1 c_cookie=15626150481215327687 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:10.198 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.196+0000 7f16e0b146c0 1 -- 10.20.193.27:0/1112766726 >> 10.20.193.27:0/1112766726 conn(0x7f16dc0ecb00 msgr2=0x7f16dc10cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:10.198 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.196+0000 7f16e0b146c0 1 -- 10.20.193.27:0/1112766726 shutdown_connections 2026-02-13T02:24:10.199 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.196+0000 7f16e0b146c0 1 -- 10.20.193.27:0/1112766726 wait complete. 2026-02-13T02:24:10.201 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.197+0000 7f16e0b146c0 1 Processor -- start 2026-02-13T02:24:10.201 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.197+0000 7f16e0b146c0 1 -- start start 2026-02-13T02:24:10.201 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.197+0000 7f16e0b146c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f16dc10f200 0x7f16dc11d330 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:10.202 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.197+0000 7f16e0b146c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f16dc115410 0x7f16dc11d870 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:10.202 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.197+0000 7f16e0b146c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f16dc122c30 0x7f16dc1230d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:10.202 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.197+0000 7f16e0b146c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f16dc123640 con 0x7f16dc10f200 2026-02-13T02:24:10.202 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.197+0000 7f16e0b146c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f16dc1237b0 con 0x7f16dc115410 2026-02-13T02:24:10.202 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.197+0000 7f16e0b146c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f16dc123970 con 0x7f16dc122c30 2026-02-13T02:24:10.202 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.197+0000 7f16da5756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f16dc10f200 0x7f16dc11d330 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:10.202 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.197+0000 7f16da5756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f16dc10f200 0x7f16dc11d330 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:47374/0 (socket says 10.20.193.27:47374) 2026-02-13T02:24:10.202 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.197+0000 7f16da5756c0 1 -- 10.20.193.27:0/1937665080 learned_addr learned my addr 10.20.193.27:0/1937665080 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:24:10.202 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.197+0000 7f16da5756c0 1 -- 10.20.193.27:0/1937665080 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f16dc122c30 msgr2=0x7f16dc1230d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2026-02-13T02:24:10.202 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.197+0000 7f16da5756c0 1 --2- 10.20.193.27:0/1937665080 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f16dc122c30 0x7f16dc1230d0 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:10.202 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.197+0000 7f16da5756c0 1 -- 10.20.193.27:0/1937665080 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f16dc115410 msgr2=0x7f16dc11d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:10.203 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.197+0000 7f16da5756c0 1 --2- 10.20.193.27:0/1937665080 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f16dc115410 0x7f16dc11d870 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:10.203 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.197+0000 7f16da5756c0 1 -- 10.20.193.27:0/1937665080 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d0009590 con 0x7f16dc10f200 2026-02-13T02:24:10.203 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.197+0000 7f16da5756c0 1 --2- 10.20.193.27:0/1937665080 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f16dc10f200 0x7f16dc11d330 secure :-1 s=READY pgs=393 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f16d002f230 tx=0x7f16d002fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:10.203 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.197+0000 7f16c37fe6c0 1 -- 10.20.193.27:0/1937665080 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f16d0002ba0 con 0x7f16dc10f200 2026-02-13T02:24:10.203 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.197+0000 7f16c37fe6c0 1 -- 10.20.193.27:0/1937665080 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f16d0002d00 con 0x7f16dc10f200 2026-02-13T02:24:10.203 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.197+0000 7f16e0b146c0 1 -- 10.20.193.27:0/1937665080 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16dc163dc0 con 0x7f16dc10f200 2026-02-13T02:24:10.203 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.197+0000 7f16c37fe6c0 1 -- 10.20.193.27:0/1937665080 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f16d00372f0 con 0x7f16dc10f200 2026-02-13T02:24:10.203 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.197+0000 7f16e0b146c0 1 -- 10.20.193.27:0/1937665080 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16dc164240 con 0x7f16dc10f200 2026-02-13T02:24:10.203 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.198+0000 7f16e0b146c0 1 -- 10.20.193.27:0/1937665080 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16dc1104d0 con 0x7f16dc10f200 2026-02-13T02:24:10.203 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.199+0000 7f16c37fe6c0 1 -- 10.20.193.27:0/1937665080 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f16d00078d0 con 0x7f16dc10f200 2026-02-13T02:24:10.203 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.200+0000 7f16c37fe6c0 1 --2- 10.20.193.27:0/1937665080 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f16b40b5680 0x7f16b40d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:10.203 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.200+0000 7f16d9d746c0 1 --2- 10.20.193.27:0/1937665080 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f16b40b5680 0x7f16b40d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:10.203 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.200+0000 7f16c37fe6c0 1 -- 10.20.193.27:0/1937665080 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f16d0107c50 con 0x7f16dc10f200 2026-02-13T02:24:10.204 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.200+0000 7f16d9d746c0 1 --2- 10.20.193.27:0/1937665080 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f16b40b5680 0x7f16b40d5b30 secure :-1 s=READY pgs=260 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f16c4005980 tx=0x7f16c4005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:10.204 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.200+0000 7f16c37fe6c0 1 -- 10.20.193.27:0/1937665080 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f16d00ce130 con 0x7f16dc10f200 2026-02-13T02:24:10.315 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.313+0000 7f16e0b146c0 1 -- 10.20.193.27:0/1937665080 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f16dc10f5d0 con 0x7f16dc10f200 2026-02-13T02:24:10.316 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:10 trial027 ceph-mon[24567]: pgmap v243: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:10.316 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.313+0000 7f16c37fe6c0 1 -- 10.20.193.27:0/1937665080 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f16d00cd880 con 0x7f16dc10f200 2026-02-13T02:24:10.316 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:24:10.317 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:24:10.317 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.314+0000 7f16e0b146c0 1 -- 10.20.193.27:0/1937665080 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f16b40b5680 msgr2=0x7f16b40d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:10.317 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.314+0000 7f16e0b146c0 1 --2- 10.20.193.27:0/1937665080 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f16b40b5680 0x7f16b40d5b30 secure :-1 s=READY pgs=260 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f16c4005980 tx=0x7f16c4005910 comp rx=0 tx=0).stop 2026-02-13T02:24:10.317 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.314+0000 7f16e0b146c0 1 -- 10.20.193.27:0/1937665080 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f16dc10f200 msgr2=0x7f16dc11d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:10.318 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.314+0000 7f16e0b146c0 1 --2- 10.20.193.27:0/1937665080 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f16dc10f200 0x7f16dc11d330 secure :-1 s=READY pgs=393 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f16d002f230 tx=0x7f16d002fd40 comp rx=0 tx=0).stop 2026-02-13T02:24:10.318 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.315+0000 7f16e0b146c0 1 -- 10.20.193.27:0/1937665080 shutdown_connections 2026-02-13T02:24:10.318 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.315+0000 7f16e0b146c0 1 --2- 10.20.193.27:0/1937665080 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f16b40b5680 0x7f16b40d5b30 unknown :-1 s=CLOSED pgs=260 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:10.318 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.315+0000 7f16e0b146c0 1 --2- 10.20.193.27:0/1937665080 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f16dc122c30 0x7f16dc1230d0 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:10.318 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.315+0000 7f16e0b146c0 1 --2- 10.20.193.27:0/1937665080 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f16dc115410 0x7f16dc11d870 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:10.318 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.315+0000 7f16e0b146c0 1 --2- 10.20.193.27:0/1937665080 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f16dc10f200 0x7f16dc11d330 unknown :-1 s=CLOSED pgs=393 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:10.318 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.315+0000 7f16e0b146c0 1 -- 10.20.193.27:0/1937665080 >> 10.20.193.27:0/1937665080 conn(0x7f16dc0ecb00 msgr2=0x7f16dc10cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:10.318 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.315+0000 7f16e0b146c0 1 -- 10.20.193.27:0/1937665080 shutdown_connections 2026-02-13T02:24:10.318 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:10.315+0000 7f16e0b146c0 1 -- 10.20.193.27:0/1937665080 wait complete. 2026-02-13T02:24:10.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:10 trial028 ceph-mon[26701]: pgmap v243: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:10.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:10 trial107 ceph-mon[27422]: pgmap v243: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:11.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:11 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1937665080' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:11.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:11 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1937665080' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:11.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:11 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1937665080' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:12.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:12 trial028 ceph-mon[26701]: pgmap v244: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:12.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:12 trial107 ceph-mon[27422]: pgmap v244: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:12.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:12 trial027 ceph-mon[24567]: pgmap v244: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:13.360 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:24:13.479 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:24:13.599 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.595+0000 7fcfb278c6c0 1 -- 10.20.193.27:0/3119684374 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcfac115410 msgr2=0x7fcfac175db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:13.600 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.595+0000 7fcfb278c6c0 1 --2- 10.20.193.27:0/3119684374 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcfac115410 0x7fcfac175db0 secure :-1 s=READY pgs=394 gs=1 cs=0 l=1 c_cookie=2414522882136881656 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcfa000aa70 tx=0x7fcfa002fde0 comp rx=0 tx=0).stop 2026-02-13T02:24:13.600 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.595+0000 7fcfb278c6c0 1 -- 10.20.193.27:0/3119684374 shutdown_connections 2026-02-13T02:24:13.600 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.595+0000 7fcfb278c6c0 1 --2- 10.20.193.27:0/3119684374 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcfac115410 0x7fcfac175db0 unknown :-1 s=CLOSED pgs=394 gs=1 cs=0 l=1 c_cookie=2414522882136881656 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:13.600 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.595+0000 7fcfb278c6c0 1 --2- 10.20.193.27:0/3119684374 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcfac10fbf0 0x7fcfac114c30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:13.600 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.595+0000 7fcfb278c6c0 1 --2- 10.20.193.27:0/3119684374 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fcfac10f200 0x7fcfac10f5d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:13.600 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.595+0000 7fcfb278c6c0 1 -- 10.20.193.27:0/3119684374 >> 10.20.193.27:0/3119684374 conn(0x7fcfac0ecb00 msgr2=0x7fcfac10cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:13.600 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.595+0000 7fcfb278c6c0 1 -- 10.20.193.27:0/3119684374 shutdown_connections 2026-02-13T02:24:13.600 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.595+0000 7fcfb278c6c0 1 -- 10.20.193.27:0/3119684374 wait complete. 2026-02-13T02:24:13.600 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.595+0000 7fcfb278c6c0 1 Processor -- start 2026-02-13T02:24:13.600 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.595+0000 7fcfb278c6c0 1 -- start start 2026-02-13T02:24:13.600 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.595+0000 7fcfb278c6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcfac10f200 0x7fcfac11d1f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:13.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.595+0000 7fcfb278c6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fcfac115410 0x7fcfac11d730 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:13.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.595+0000 7fcfb278c6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcfac122af0 0x7fcfac122f90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:13.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.595+0000 7fcfb278c6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac123500 con 0x7fcfac10f200 2026-02-13T02:24:13.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.595+0000 7fcfb278c6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac123670 con 0x7fcfac122af0 2026-02-13T02:24:13.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.595+0000 7fcfb278c6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac123830 con 0x7fcfac115410 2026-02-13T02:24:13.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.595+0000 7fcfabfff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcfac10f200 0x7fcfac11d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:13.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.596+0000 7fcfabfff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcfac10f200 0x7fcfac11d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:47396/0 (socket says 10.20.193.27:47396) 2026-02-13T02:24:13.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.596+0000 7fcfabfff6c0 1 -- 10.20.193.27:0/1112907562 learned_addr learned my addr 10.20.193.27:0/1112907562 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:24:13.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.596+0000 7fcfb0d026c0 1 --2- 10.20.193.27:0/1112907562 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcfac122af0 0x7fcfac122f90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:13.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.596+0000 7fcfab7fe6c0 1 --2- 10.20.193.27:0/1112907562 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fcfac115410 0x7fcfac11d730 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:13.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.596+0000 7fcfabfff6c0 1 -- 10.20.193.27:0/1112907562 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fcfac115410 msgr2=0x7fcfac11d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:13.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.596+0000 7fcfabfff6c0 1 --2- 10.20.193.27:0/1112907562 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fcfac115410 0x7fcfac11d730 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:13.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.596+0000 7fcfabfff6c0 1 -- 10.20.193.27:0/1112907562 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcfac122af0 msgr2=0x7fcfac122f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:13.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.596+0000 7fcfabfff6c0 1 --2- 10.20.193.27:0/1112907562 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcfac122af0 0x7fcfac122f90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:13.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.596+0000 7fcfabfff6c0 1 -- 10.20.193.27:0/1112907562 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa000a660 con 0x7fcfac10f200 2026-02-13T02:24:13.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.596+0000 7fcfb0d026c0 1 --2- 10.20.193.27:0/1112907562 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcfac122af0 0x7fcfac122f90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:24:13.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.596+0000 7fcfabfff6c0 1 --2- 10.20.193.27:0/1112907562 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcfac10f200 0x7fcfac11d1f0 secure :-1 s=READY pgs=395 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcf9800b710 tx=0x7fcf9800bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:13.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.596+0000 7fcfa97fa6c0 1 -- 10.20.193.27:0/1112907562 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fcf98004250 con 0x7fcfac10f200 2026-02-13T02:24:13.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.596+0000 7fcfa97fa6c0 1 -- 10.20.193.27:0/1112907562 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fcf98009070 con 0x7fcfac10f200 2026-02-13T02:24:13.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.596+0000 7fcfa97fa6c0 1 -- 10.20.193.27:0/1112907562 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fcf9800ccd0 con 0x7fcfac10f200 2026-02-13T02:24:13.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.596+0000 7fcfb278c6c0 1 -- 10.20.193.27:0/1112907562 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfac163ce0 con 0x7fcfac10f200 2026-02-13T02:24:13.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.596+0000 7fcfb278c6c0 1 -- 10.20.193.27:0/1112907562 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfac164170 con 0x7fcfac10f200 2026-02-13T02:24:13.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.596+0000 7fcfb278c6c0 1 -- 10.20.193.27:0/1112907562 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfac1104d0 con 0x7fcfac10f200 2026-02-13T02:24:13.604 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.597+0000 7fcfa97fa6c0 1 -- 10.20.193.27:0/1112907562 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7fcf980028c0 con 0x7fcfac10f200 2026-02-13T02:24:13.604 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.598+0000 7fcfa97fa6c0 1 --2- 10.20.193.27:0/1112907562 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fcf800b5680 0x7fcf800d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:13.604 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.598+0000 7fcfab7fe6c0 1 --2- 10.20.193.27:0/1112907562 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fcf800b5680 0x7fcf800d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:13.604 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.598+0000 7fcfa97fa6c0 1 -- 10.20.193.27:0/1112907562 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7fcf980e3c20 con 0x7fcfac10f200 2026-02-13T02:24:13.604 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.599+0000 7fcfab7fe6c0 1 --2- 10.20.193.27:0/1112907562 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fcf800b5680 0x7fcf800d5b30 secure :-1 s=READY pgs=261 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcfac123b80 tx=0x7fcf9c005d20 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:13.604 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.599+0000 7fcfa97fa6c0 1 -- 10.20.193.27:0/1112907562 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fcf980a66a0 con 0x7fcfac10f200 2026-02-13T02:24:13.718 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.715+0000 7fcfb278c6c0 1 -- 10.20.193.27:0/1112907562 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fcfac10f5d0 con 0x7fcfac10f200 2026-02-13T02:24:13.718 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.716+0000 7fcfa97fa6c0 1 -- 10.20.193.27:0/1112907562 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7fcf9800ce60 con 0x7fcfac10f200 2026-02-13T02:24:13.718 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:24:13.719 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:24:13.720 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.717+0000 7fcfb278c6c0 1 -- 10.20.193.27:0/1112907562 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fcf800b5680 msgr2=0x7fcf800d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:13.720 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.717+0000 7fcfb278c6c0 1 --2- 10.20.193.27:0/1112907562 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fcf800b5680 0x7fcf800d5b30 secure :-1 s=READY pgs=261 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcfac123b80 tx=0x7fcf9c005d20 comp rx=0 tx=0).stop 2026-02-13T02:24:13.720 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.717+0000 7fcfb278c6c0 1 -- 10.20.193.27:0/1112907562 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcfac10f200 msgr2=0x7fcfac11d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:13.720 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.717+0000 7fcfb278c6c0 1 --2- 10.20.193.27:0/1112907562 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcfac10f200 0x7fcfac11d1f0 secure :-1 s=READY pgs=395 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcf9800b710 tx=0x7fcf9800bbe0 comp rx=0 tx=0).stop 2026-02-13T02:24:13.720 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.717+0000 7fcfb278c6c0 1 -- 10.20.193.27:0/1112907562 shutdown_connections 2026-02-13T02:24:13.720 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.717+0000 7fcfb278c6c0 1 --2- 10.20.193.27:0/1112907562 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fcf800b5680 0x7fcf800d5b30 unknown :-1 s=CLOSED pgs=261 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:13.720 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.717+0000 7fcfb278c6c0 1 --2- 10.20.193.27:0/1112907562 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcfac122af0 0x7fcfac122f90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:13.720 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.717+0000 7fcfb278c6c0 1 --2- 10.20.193.27:0/1112907562 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fcfac115410 0x7fcfac11d730 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:13.721 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.717+0000 7fcfb278c6c0 1 --2- 10.20.193.27:0/1112907562 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcfac10f200 0x7fcfac11d1f0 unknown :-1 s=CLOSED pgs=395 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:13.721 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.717+0000 7fcfb278c6c0 1 -- 10.20.193.27:0/1112907562 >> 10.20.193.27:0/1112907562 conn(0x7fcfac0ecb00 msgr2=0x7fcfac10cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:13.721 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.717+0000 7fcfb278c6c0 1 -- 10.20.193.27:0/1112907562 shutdown_connections 2026-02-13T02:24:13.721 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:13.717+0000 7fcfb278c6c0 1 -- 10.20.193.27:0/1112907562 wait complete. 2026-02-13T02:24:14.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:14 trial028 ceph-mon[26701]: pgmap v245: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:24:14.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:14 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1112907562' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:14.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:14 trial107 ceph-mon[27422]: pgmap v245: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:24:14.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:14 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1112907562' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:14.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:14 trial027 ceph-mon[24567]: pgmap v245: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:24:14.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:14 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1112907562' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:16.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:16 trial028 ceph-mon[26701]: pgmap v246: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:16.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:16 trial107 ceph-mon[27422]: pgmap v246: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:16.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:16 trial027 ceph-mon[24567]: pgmap v246: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:16.759 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:24:16.879 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:24:17.016 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.011+0000 7fc10d79b6c0 1 -- 10.20.193.27:0/614851279 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc10810cfb0 msgr2=0x7fc10810d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:17.016 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.011+0000 7fc10d79b6c0 1 --2- 10.20.193.27:0/614851279 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc10810cfb0 0x7fc10810d380 secure :-1 s=READY pgs=396 gs=1 cs=0 l=1 c_cookie=8614109613957994218 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc0fc0099a0 tx=0x7fc0fc02ed00 comp rx=0 tx=0).stop 2026-02-13T02:24:17.016 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.011+0000 7fc10d79b6c0 1 -- 10.20.193.27:0/614851279 shutdown_connections 2026-02-13T02:24:17.016 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.011+0000 7fc10d79b6c0 1 --2- 10.20.193.27:0/614851279 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc1081131c0 0x7fc108173b60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:17.016 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.011+0000 7fc10d79b6c0 1 --2- 10.20.193.27:0/614851279 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc10810d9a0 0x7fc1081129e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:17.016 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.011+0000 7fc10d79b6c0 1 --2- 10.20.193.27:0/614851279 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc10810cfb0 0x7fc10810d380 unknown :-1 s=CLOSED pgs=396 gs=1 cs=0 l=1 c_cookie=8614109613957994218 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:17.016 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.011+0000 7fc10d79b6c0 1 -- 10.20.193.27:0/614851279 >> 10.20.193.27:0/614851279 conn(0x7fc1080ea8b0 msgr2=0x7fc10810acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:17.016 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.011+0000 7fc10d79b6c0 1 -- 10.20.193.27:0/614851279 shutdown_connections 2026-02-13T02:24:17.016 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.011+0000 7fc10d79b6c0 1 -- 10.20.193.27:0/614851279 wait complete. 2026-02-13T02:24:17.017 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.012+0000 7fc10d79b6c0 1 Processor -- start 2026-02-13T02:24:17.017 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.012+0000 7fc10d79b6c0 1 -- start start 2026-02-13T02:24:17.017 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.012+0000 7fc10d79b6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc10810cfb0 0x7fc1080f4c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:17.017 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.012+0000 7fc10d79b6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc1081131c0 0x7fc108173a60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:17.017 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.012+0000 7fc10d79b6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc1080f52c0 0x7fc1080fa590 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:17.017 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.012+0000 7fc10d79b6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc1080f5920 con 0x7fc1081131c0 2026-02-13T02:24:17.017 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.012+0000 7fc10d79b6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc1080faad0 con 0x7fc1080f52c0 2026-02-13T02:24:17.017 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.012+0000 7fc10d79b6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc1080fac90 con 0x7fc10810cfb0 2026-02-13T02:24:17.017 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.012+0000 7fc1067fc6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc1081131c0 0x7fc108173a60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:17.017 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.012+0000 7fc1067fc6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc1081131c0 0x7fc108173a60 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:47416/0 (socket says 10.20.193.27:47416) 2026-02-13T02:24:17.017 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.012+0000 7fc1067fc6c0 1 -- 10.20.193.27:0/3133532393 learned_addr learned my addr 10.20.193.27:0/3133532393 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:24:17.017 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.012+0000 7fc1077fe6c0 1 --2- 10.20.193.27:0/3133532393 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc1080f52c0 0x7fc1080fa590 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:17.018 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.012+0000 7fc106ffd6c0 1 --2- 10.20.193.27:0/3133532393 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc10810cfb0 0x7fc1080f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:17.018 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.012+0000 7fc1067fc6c0 1 -- 10.20.193.27:0/3133532393 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc10810cfb0 msgr2=0x7fc1080f4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:17.018 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.012+0000 7fc1067fc6c0 1 --2- 10.20.193.27:0/3133532393 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc10810cfb0 0x7fc1080f4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:17.018 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.012+0000 7fc1067fc6c0 1 -- 10.20.193.27:0/3133532393 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc1080f52c0 msgr2=0x7fc1080fa590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:17.018 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.012+0000 7fc1067fc6c0 1 --2- 10.20.193.27:0/3133532393 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc1080f52c0 0x7fc1080fa590 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:17.018 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.012+0000 7fc1067fc6c0 1 -- 10.20.193.27:0/3133532393 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0fc009590 con 0x7fc1081131c0 2026-02-13T02:24:17.018 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.012+0000 7fc1067fc6c0 1 --2- 10.20.193.27:0/3133532393 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc1081131c0 0x7fc108173a60 secure :-1 s=READY pgs=397 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc0f0005fd0 tx=0x7fc0f000bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:17.018 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.012+0000 7fc0e7fff6c0 1 -- 10.20.193.27:0/3133532393 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc0f0004280 con 0x7fc1081131c0 2026-02-13T02:24:17.018 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.012+0000 7fc0e7fff6c0 1 -- 10.20.193.27:0/3133532393 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc0f00043e0 con 0x7fc1081131c0 2026-02-13T02:24:17.018 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.012+0000 7fc10d79b6c0 1 -- 10.20.193.27:0/3133532393 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1080faf10 con 0x7fc1081131c0 2026-02-13T02:24:17.018 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.012+0000 7fc0e7fff6c0 1 -- 10.20.193.27:0/3133532393 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc0f000ccd0 con 0x7fc1081131c0 2026-02-13T02:24:17.018 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.012+0000 7fc10d79b6c0 1 -- 10.20.193.27:0/3133532393 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1080fb700 con 0x7fc1081131c0 2026-02-13T02:24:17.019 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.013+0000 7fc10d79b6c0 1 -- 10.20.193.27:0/3133532393 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc10810e280 con 0x7fc1081131c0 2026-02-13T02:24:17.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.014+0000 7fc0e7fff6c0 1 -- 10.20.193.27:0/3133532393 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7fc0f00026b0 con 0x7fc1081131c0 2026-02-13T02:24:17.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.015+0000 7fc0e7fff6c0 1 --2- 10.20.193.27:0/3133532393 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc0dc0b5680 0x7fc0dc0d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:17.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.015+0000 7fc106ffd6c0 1 --2- 10.20.193.27:0/3133532393 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc0dc0b5680 0x7fc0dc0d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:17.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.015+0000 7fc0e7fff6c0 1 -- 10.20.193.27:0/3133532393 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7fc0f00e39f0 con 0x7fc1081131c0 2026-02-13T02:24:17.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.015+0000 7fc0e7fff6c0 1 -- 10.20.193.27:0/3133532393 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fc0f011c5e0 con 0x7fc1081131c0 2026-02-13T02:24:17.020 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.015+0000 7fc106ffd6c0 1 --2- 10.20.193.27:0/3133532393 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc0dc0b5680 0x7fc0dc0d5b30 secure :-1 s=READY pgs=262 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc0fc0023e0 tx=0x7fc0fc031f90 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:17.135 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.132+0000 7fc10d79b6c0 1 -- 10.20.193.27:0/3133532393 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc1080f5de0 con 0x7fc1081131c0 2026-02-13T02:24:17.135 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.132+0000 7fc0e7fff6c0 1 -- 10.20.193.27:0/3133532393 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7fc0f00a6470 con 0x7fc1081131c0 2026-02-13T02:24:17.135 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:24:17.135 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:24:17.137 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.134+0000 7fc10d79b6c0 1 -- 10.20.193.27:0/3133532393 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc0dc0b5680 msgr2=0x7fc0dc0d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:17.137 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.134+0000 7fc10d79b6c0 1 --2- 10.20.193.27:0/3133532393 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc0dc0b5680 0x7fc0dc0d5b30 secure :-1 s=READY pgs=262 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc0fc0023e0 tx=0x7fc0fc031f90 comp rx=0 tx=0).stop 2026-02-13T02:24:17.137 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.134+0000 7fc10d79b6c0 1 -- 10.20.193.27:0/3133532393 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc1081131c0 msgr2=0x7fc108173a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:17.137 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.134+0000 7fc10d79b6c0 1 --2- 10.20.193.27:0/3133532393 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc1081131c0 0x7fc108173a60 secure :-1 s=READY pgs=397 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc0f0005fd0 tx=0x7fc0f000bca0 comp rx=0 tx=0).stop 2026-02-13T02:24:17.137 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.134+0000 7fc10d79b6c0 1 -- 10.20.193.27:0/3133532393 shutdown_connections 2026-02-13T02:24:17.137 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.134+0000 7fc10d79b6c0 1 --2- 10.20.193.27:0/3133532393 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc0dc0b5680 0x7fc0dc0d5b30 unknown :-1 s=CLOSED pgs=262 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:17.137 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.134+0000 7fc10d79b6c0 1 --2- 10.20.193.27:0/3133532393 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc1080f52c0 0x7fc1080fa590 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:17.138 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.134+0000 7fc10d79b6c0 1 --2- 10.20.193.27:0/3133532393 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc1081131c0 0x7fc108173a60 unknown :-1 s=CLOSED pgs=397 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:17.138 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.134+0000 7fc10d79b6c0 1 --2- 10.20.193.27:0/3133532393 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc10810cfb0 0x7fc1080f4c90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:17.138 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.134+0000 7fc10d79b6c0 1 -- 10.20.193.27:0/3133532393 >> 10.20.193.27:0/3133532393 conn(0x7fc1080ea8b0 msgr2=0x7fc1080605e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:17.138 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.134+0000 7fc10d79b6c0 1 -- 10.20.193.27:0/3133532393 shutdown_connections 2026-02-13T02:24:17.138 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:17.134+0000 7fc10d79b6c0 1 -- 10.20.193.27:0/3133532393 wait complete. 2026-02-13T02:24:17.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:17 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:24:17.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:17 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3133532393' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:17.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:17 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:24:17.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:17 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3133532393' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:17.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:17 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:24:17.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:17 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3133532393' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:18.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:18 trial028 ceph-mon[26701]: pgmap v247: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:24:18.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:18 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:24:18.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:18 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:24:18.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:18 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:24:18.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:18 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:24:18.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:18 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:24:18.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:18 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:24:18.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:18 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:24:18.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:18 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:24:18.495 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:18 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:24:18.495 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:18 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:24:18.495 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:18 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:24:18.495 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:18 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:24:18.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:18 trial107 ceph-mon[27422]: pgmap v247: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:24:18.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:18 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:24:18.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:18 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:24:18.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:18 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:24:18.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:18 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:24:18.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:18 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:24:18.502 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:18 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:24:18.502 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:18 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:24:18.502 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:18 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:24:18.502 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:18 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:24:18.502 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:18 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:24:18.502 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:18 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:24:18.502 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:18 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:24:18.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:18 trial027 ceph-mon[24567]: pgmap v247: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:24:18.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:18 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:24:18.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:18 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:24:18.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:18 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:24:18.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:18 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:24:18.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:18 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:24:18.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:18 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:24:18.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:18 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:24:18.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:18 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:24:18.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:18 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:24:18.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:18 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:24:18.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:18 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:24:18.660 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:18 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:24:19.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:19 trial028 ceph-mon[26701]: pgmap v248: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 915 B/s rd, 1 op/s 2026-02-13T02:24:19.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:19 trial028 ceph-mon[26701]: pgmap v249: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 2 op/s 2026-02-13T02:24:19.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:19 trial107 ceph-mon[27422]: pgmap v248: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 915 B/s rd, 1 op/s 2026-02-13T02:24:19.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:19 trial107 ceph-mon[27422]: pgmap v249: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 2 op/s 2026-02-13T02:24:19.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:19 trial027 ceph-mon[24567]: pgmap v248: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 915 B/s rd, 1 op/s 2026-02-13T02:24:19.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:19 trial027 ceph-mon[24567]: pgmap v249: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 2 op/s 2026-02-13T02:24:20.195 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:24:20.314 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:24:20.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.428+0000 7fcb853db6c0 1 -- 10.20.193.27:0/3727290549 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcb8010f200 msgr2=0x7fcb8010f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:20.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.428+0000 7fcb853db6c0 1 --2- 10.20.193.27:0/3727290549 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcb8010f200 0x7fcb8010f5d0 secure :-1 s=READY pgs=398 gs=1 cs=0 l=1 c_cookie=16076689773368493584 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcb740099a0 tx=0x7fcb7402ed20 comp rx=0 tx=0).stop 2026-02-13T02:24:20.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.428+0000 7fcb853db6c0 1 -- 10.20.193.27:0/3727290549 shutdown_connections 2026-02-13T02:24:20.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.428+0000 7fcb853db6c0 1 --2- 10.20.193.27:0/3727290549 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fcb80115410 0x7fcb80175db0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:20.433 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.428+0000 7fcb853db6c0 1 --2- 10.20.193.27:0/3727290549 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcb8010fbf0 0x7fcb80114c30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:20.434 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.428+0000 7fcb853db6c0 1 --2- 10.20.193.27:0/3727290549 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcb8010f200 0x7fcb8010f5d0 unknown :-1 s=CLOSED pgs=398 gs=1 cs=0 l=1 c_cookie=16076689773368493584 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:20.434 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.428+0000 7fcb853db6c0 1 -- 10.20.193.27:0/3727290549 >> 10.20.193.27:0/3727290549 conn(0x7fcb800ecb00 msgr2=0x7fcb8010cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:20.434 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.428+0000 7fcb853db6c0 1 -- 10.20.193.27:0/3727290549 shutdown_connections 2026-02-13T02:24:20.434 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.428+0000 7fcb853db6c0 1 -- 10.20.193.27:0/3727290549 wait complete. 2026-02-13T02:24:20.434 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.429+0000 7fcb853db6c0 1 Processor -- start 2026-02-13T02:24:20.434 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.429+0000 7fcb853db6c0 1 -- start start 2026-02-13T02:24:20.434 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.429+0000 7fcb853db6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fcb8010f200 0x7fcb8011d230 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:20.434 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.429+0000 7fcb853db6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcb80115410 0x7fcb8011d770 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:20.434 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.429+0000 7fcb853db6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcb80122b30 0x7fcb80122fd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:20.434 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.429+0000 7fcb853db6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcb80123540 con 0x7fcb80115410 2026-02-13T02:24:20.434 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.429+0000 7fcb853db6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fcb801236b0 con 0x7fcb80122b30 2026-02-13T02:24:20.435 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.429+0000 7fcb853db6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcb80123870 con 0x7fcb8010f200 2026-02-13T02:24:20.435 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.429+0000 7fcb7e7fc6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcb80115410 0x7fcb8011d770 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:20.435 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.429+0000 7fcb7e7fc6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcb80115410 0x7fcb8011d770 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:47380/0 (socket says 10.20.193.27:47380) 2026-02-13T02:24:20.435 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.429+0000 7fcb7e7fc6c0 1 -- 10.20.193.27:0/4133543782 learned_addr learned my addr 10.20.193.27:0/4133543782 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:24:20.435 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.429+0000 7fcb7effd6c0 1 --2- 10.20.193.27:0/4133543782 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fcb8010f200 0x7fcb8011d230 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:20.435 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.429+0000 7fcb7f7fe6c0 1 --2- 10.20.193.27:0/4133543782 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcb80122b30 0x7fcb80122fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:20.435 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.429+0000 7fcb7e7fc6c0 1 -- 10.20.193.27:0/4133543782 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fcb8010f200 msgr2=0x7fcb8011d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:20.435 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.429+0000 7fcb7e7fc6c0 1 --2- 10.20.193.27:0/4133543782 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fcb8010f200 0x7fcb8011d230 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:20.435 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.429+0000 7fcb7e7fc6c0 1 -- 10.20.193.27:0/4133543782 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcb80122b30 msgr2=0x7fcb80122fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:20.435 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.429+0000 7fcb7e7fc6c0 1 --2- 10.20.193.27:0/4133543782 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcb80122b30 0x7fcb80122fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:20.435 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.429+0000 7fcb7e7fc6c0 1 -- 10.20.193.27:0/4133543782 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb74009590 con 0x7fcb80115410 2026-02-13T02:24:20.435 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.429+0000 7fcb7e7fc6c0 1 --2- 10.20.193.27:0/4133543782 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcb80115410 0x7fcb8011d770 secure :-1 s=READY pgs=399 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcb6800b6c0 tx=0x7fcb6800bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:20.436 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.429+0000 7fcb5ffff6c0 1 -- 10.20.193.27:0/4133543782 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fcb680044a0 con 0x7fcb80115410 2026-02-13T02:24:20.436 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.429+0000 7fcb853db6c0 1 -- 10.20.193.27:0/4133543782 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb80163d20 con 0x7fcb80115410 2026-02-13T02:24:20.436 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.429+0000 7fcb5ffff6c0 1 -- 10.20.193.27:0/4133543782 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fcb6800be50 con 0x7fcb80115410 2026-02-13T02:24:20.436 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.429+0000 7fcb5ffff6c0 1 -- 10.20.193.27:0/4133543782 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fcb6800cc90 con 0x7fcb80115410 2026-02-13T02:24:20.436 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.429+0000 7fcb853db6c0 1 -- 10.20.193.27:0/4133543782 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb80164230 con 0x7fcb80115410 2026-02-13T02:24:20.436 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.430+0000 7fcb853db6c0 1 -- 10.20.193.27:0/4133543782 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb801104d0 con 0x7fcb80115410 2026-02-13T02:24:20.436 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.431+0000 7fcb5ffff6c0 1 -- 10.20.193.27:0/4133543782 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7fcb680026b0 con 0x7fcb80115410 2026-02-13T02:24:20.437 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.432+0000 7fcb5ffff6c0 1 --2- 10.20.193.27:0/4133543782 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fcb540b5600 0x7fcb540d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:20.437 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.432+0000 7fcb7effd6c0 1 --2- 10.20.193.27:0/4133543782 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fcb540b5600 0x7fcb540d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:20.437 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.432+0000 7fcb5ffff6c0 1 -- 10.20.193.27:0/4133543782 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7fcb680e38d0 con 0x7fcb80115410 2026-02-13T02:24:20.437 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.432+0000 7fcb5ffff6c0 1 -- 10.20.193.27:0/4133543782 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fcb680a9db0 con 0x7fcb80115410 2026-02-13T02:24:20.437 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.432+0000 7fcb7effd6c0 1 --2- 10.20.193.27:0/4133543782 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fcb540b5600 0x7fcb540d5ab0 secure :-1 s=READY pgs=263 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcb74031fd0 tx=0x7fcb740023a0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:20.547 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.545+0000 7fcb853db6c0 1 -- 10.20.193.27:0/4133543782 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fcb8010f5d0 con 0x7fcb80115410 2026-02-13T02:24:20.548 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:24:20.548 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:24:20.549 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.545+0000 7fcb5ffff6c0 1 -- 10.20.193.27:0/4133543782 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7fcb680a9500 con 0x7fcb80115410 2026-02-13T02:24:20.549 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.546+0000 7fcb853db6c0 1 -- 10.20.193.27:0/4133543782 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fcb540b5600 msgr2=0x7fcb540d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:20.549 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.546+0000 7fcb853db6c0 1 --2- 10.20.193.27:0/4133543782 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fcb540b5600 0x7fcb540d5ab0 secure :-1 s=READY pgs=263 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcb74031fd0 tx=0x7fcb740023a0 comp rx=0 tx=0).stop 2026-02-13T02:24:20.549 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.546+0000 7fcb853db6c0 1 -- 10.20.193.27:0/4133543782 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcb80115410 msgr2=0x7fcb8011d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:20.550 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.546+0000 7fcb853db6c0 1 --2- 10.20.193.27:0/4133543782 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcb80115410 0x7fcb8011d770 secure :-1 s=READY pgs=399 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcb6800b6c0 tx=0x7fcb6800bb90 comp rx=0 tx=0).stop 2026-02-13T02:24:20.550 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.546+0000 7fcb853db6c0 1 -- 10.20.193.27:0/4133543782 shutdown_connections 2026-02-13T02:24:20.550 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.546+0000 7fcb853db6c0 1 --2- 10.20.193.27:0/4133543782 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fcb540b5600 0x7fcb540d5ab0 unknown :-1 s=CLOSED pgs=263 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:20.550 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.546+0000 7fcb853db6c0 1 --2- 10.20.193.27:0/4133543782 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcb80122b30 0x7fcb80122fd0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:20.550 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.546+0000 7fcb853db6c0 1 --2- 10.20.193.27:0/4133543782 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fcb80115410 0x7fcb8011d770 unknown :-1 s=CLOSED pgs=399 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:20.550 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.546+0000 7fcb853db6c0 1 --2- 10.20.193.27:0/4133543782 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fcb8010f200 0x7fcb8011d230 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:20.550 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.546+0000 7fcb853db6c0 1 -- 10.20.193.27:0/4133543782 >> 10.20.193.27:0/4133543782 conn(0x7fcb800ecb00 msgr2=0x7fcb8010cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:20.550 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.546+0000 7fcb853db6c0 1 -- 10.20.193.27:0/4133543782 shutdown_connections 2026-02-13T02:24:20.550 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:20.546+0000 7fcb853db6c0 1 -- 10.20.193.27:0/4133543782 wait complete. 2026-02-13T02:24:20.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:20 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/4133543782' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:20.909 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:20 trial027 ceph-mon[24567]: pgmap v250: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 2 op/s 2026-02-13T02:24:20.993 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:20 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/4133543782' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:20.993 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:20 trial028 ceph-mon[26701]: pgmap v250: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 2 op/s 2026-02-13T02:24:21.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:20 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/4133543782' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:21.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:20 trial107 ceph-mon[27422]: pgmap v250: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 2 op/s 2026-02-13T02:24:22.908 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:22 trial027 ceph-mon[24567]: pgmap v251: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 2 op/s 2026-02-13T02:24:22.993 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:22 trial028 ceph-mon[26701]: pgmap v251: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 2 op/s 2026-02-13T02:24:23.001 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:22 trial107 ceph-mon[27422]: pgmap v251: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 2 op/s 2026-02-13T02:24:23.590 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:24:23.709 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:24:23.829 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.825+0000 7fe8eb5206c0 1 -- 10.20.193.27:0/553042850 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe8e410f200 msgr2=0x7fe8e410f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:23.830 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.825+0000 7fe8eb5206c0 1 --2- 10.20.193.27:0/553042850 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe8e410f200 0x7fe8e410f5d0 secure :-1 s=READY pgs=400 gs=1 cs=0 l=1 c_cookie=12974793228192893692 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe8d40099a0 tx=0x7fe8d402ed20 comp rx=0 tx=0).stop 2026-02-13T02:24:23.830 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.825+0000 7fe8eb5206c0 1 -- 10.20.193.27:0/553042850 shutdown_connections 2026-02-13T02:24:23.830 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.825+0000 7fe8eb5206c0 1 --2- 10.20.193.27:0/553042850 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe8e4115410 0x7fe8e4175db0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:23.830 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.825+0000 7fe8eb5206c0 1 --2- 10.20.193.27:0/553042850 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe8e410fbf0 0x7fe8e4114c30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:23.830 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.825+0000 7fe8eb5206c0 1 --2- 10.20.193.27:0/553042850 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe8e410f200 0x7fe8e410f5d0 unknown :-1 s=CLOSED pgs=400 gs=1 cs=0 l=1 c_cookie=12974793228192893692 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:23.830 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.825+0000 7fe8eb5206c0 1 -- 10.20.193.27:0/553042850 >> 10.20.193.27:0/553042850 conn(0x7fe8e40ecb00 msgr2=0x7fe8e410cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:23.830 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.825+0000 7fe8eb5206c0 1 -- 10.20.193.27:0/553042850 shutdown_connections 2026-02-13T02:24:23.830 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.825+0000 7fe8eb5206c0 1 -- 10.20.193.27:0/553042850 wait complete. 2026-02-13T02:24:23.830 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.825+0000 7fe8eb5206c0 1 Processor -- start 2026-02-13T02:24:23.830 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.826+0000 7fe8eb5206c0 1 -- start start 2026-02-13T02:24:23.831 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.826+0000 7fe8eb5206c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe8e410f200 0x7fe8e411d260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:23.831 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.826+0000 7fe8eb5206c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe8e4115410 0x7fe8e411d7a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:23.831 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.826+0000 7fe8eb5206c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe8e4122b60 0x7fe8e4123000 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:23.831 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.826+0000 7fe8eb5206c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e4123570 con 0x7fe8e4122b60 2026-02-13T02:24:23.831 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.826+0000 7fe8eb5206c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e41236e0 con 0x7fe8e4115410 2026-02-13T02:24:23.831 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.826+0000 7fe8eb5206c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e41238a0 con 0x7fe8e410f200 2026-02-13T02:24:23.831 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.826+0000 7fe8e9a926c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe8e4122b60 0x7fe8e4123000 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:23.831 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.826+0000 7fe8e9a926c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe8e4122b60 0x7fe8e4123000 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:47392/0 (socket says 10.20.193.27:47392) 2026-02-13T02:24:23.831 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.826+0000 7fe8e9a926c0 1 -- 10.20.193.27:0/2756155508 learned_addr learned my addr 10.20.193.27:0/2756155508 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:24:23.831 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.826+0000 7fe8e92916c0 1 --2- 10.20.193.27:0/2756155508 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe8e410f200 0x7fe8e411d260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:23.831 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.826+0000 7fe8e8a906c0 1 --2- 10.20.193.27:0/2756155508 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe8e4115410 0x7fe8e411d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:23.831 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.826+0000 7fe8e9a926c0 1 -- 10.20.193.27:0/2756155508 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe8e410f200 msgr2=0x7fe8e411d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:23.832 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.826+0000 7fe8e9a926c0 1 --2- 10.20.193.27:0/2756155508 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe8e410f200 0x7fe8e411d260 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:23.832 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.826+0000 7fe8e9a926c0 1 -- 10.20.193.27:0/2756155508 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe8e4115410 msgr2=0x7fe8e411d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:23.832 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.826+0000 7fe8e9a926c0 1 --2- 10.20.193.27:0/2756155508 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe8e4115410 0x7fe8e411d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:23.832 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.826+0000 7fe8e9a926c0 1 -- 10.20.193.27:0/2756155508 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d4009590 con 0x7fe8e4122b60 2026-02-13T02:24:23.832 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.826+0000 7fe8e9a926c0 1 --2- 10.20.193.27:0/2756155508 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe8e4122b60 0x7fe8e4123000 secure :-1 s=READY pgs=401 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe8e0006ea0 tx=0x7fe8e000ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:23.832 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.826+0000 7fe8d27fc6c0 1 -- 10.20.193.27:0/2756155508 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe8e0003c40 con 0x7fe8e4122b60 2026-02-13T02:24:23.832 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.826+0000 7fe8d27fc6c0 1 -- 10.20.193.27:0/2756155508 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fe8e0003da0 con 0x7fe8e4122b60 2026-02-13T02:24:23.832 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.826+0000 7fe8d27fc6c0 1 -- 10.20.193.27:0/2756155508 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe8e00081c0 con 0x7fe8e4122b60 2026-02-13T02:24:23.832 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.826+0000 7fe8eb5206c0 1 -- 10.20.193.27:0/2756155508 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e4163d50 con 0x7fe8e4122b60 2026-02-13T02:24:23.832 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.826+0000 7fe8eb5206c0 1 -- 10.20.193.27:0/2756155508 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e41641e0 con 0x7fe8e4122b60 2026-02-13T02:24:23.832 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.827+0000 7fe8eb5206c0 1 -- 10.20.193.27:0/2756155508 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8e41104d0 con 0x7fe8e4122b60 2026-02-13T02:24:23.833 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.828+0000 7fe8d27fc6c0 1 -- 10.20.193.27:0/2756155508 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7fe8e0005860 con 0x7fe8e4122b60 2026-02-13T02:24:23.834 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.829+0000 7fe8d27fc6c0 1 --2- 10.20.193.27:0/2756155508 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe8c00b5680 0x7fe8c00d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:23.834 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.829+0000 7fe8e92916c0 1 --2- 10.20.193.27:0/2756155508 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe8c00b5680 0x7fe8c00d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:23.834 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.829+0000 7fe8d27fc6c0 1 -- 10.20.193.27:0/2756155508 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7fe8e00e4a60 con 0x7fe8e4122b60 2026-02-13T02:24:23.834 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.829+0000 7fe8d27fc6c0 1 -- 10.20.193.27:0/2756155508 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fe8e011d5e0 con 0x7fe8e4122b60 2026-02-13T02:24:23.834 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.829+0000 7fe8e92916c0 1 --2- 10.20.193.27:0/2756155508 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe8c00b5680 0x7fe8c00d5b30 secure :-1 s=READY pgs=264 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe8d4005b40 tx=0x7fe8d40023a0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:23.946 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.944+0000 7fe8eb5206c0 1 -- 10.20.193.27:0/2756155508 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe8e410f5d0 con 0x7fe8e4122b60 2026-02-13T02:24:23.946 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.944+0000 7fe8d27fc6c0 1 -- 10.20.193.27:0/2756155508 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7fe8e00a7520 con 0x7fe8e4122b60 2026-02-13T02:24:23.946 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:24:23.947 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:24:23.948 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.945+0000 7fe8eb5206c0 1 -- 10.20.193.27:0/2756155508 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe8c00b5680 msgr2=0x7fe8c00d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:23.948 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.945+0000 7fe8eb5206c0 1 --2- 10.20.193.27:0/2756155508 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe8c00b5680 0x7fe8c00d5b30 secure :-1 s=READY pgs=264 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe8d4005b40 tx=0x7fe8d40023a0 comp rx=0 tx=0).stop 2026-02-13T02:24:23.948 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.945+0000 7fe8eb5206c0 1 -- 10.20.193.27:0/2756155508 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe8e4122b60 msgr2=0x7fe8e4123000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:23.948 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.945+0000 7fe8eb5206c0 1 --2- 10.20.193.27:0/2756155508 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe8e4122b60 0x7fe8e4123000 secure :-1 s=READY pgs=401 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe8e0006ea0 tx=0x7fe8e000ce50 comp rx=0 tx=0).stop 2026-02-13T02:24:23.948 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.945+0000 7fe8eb5206c0 1 -- 10.20.193.27:0/2756155508 shutdown_connections 2026-02-13T02:24:23.948 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.945+0000 7fe8eb5206c0 1 --2- 10.20.193.27:0/2756155508 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe8c00b5680 0x7fe8c00d5b30 unknown :-1 s=CLOSED pgs=264 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:23.948 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.945+0000 7fe8eb5206c0 1 --2- 10.20.193.27:0/2756155508 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe8e4122b60 0x7fe8e4123000 unknown :-1 s=CLOSED pgs=401 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:23.948 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.945+0000 7fe8eb5206c0 1 --2- 10.20.193.27:0/2756155508 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe8e4115410 0x7fe8e411d7a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:23.948 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.945+0000 7fe8eb5206c0 1 --2- 10.20.193.27:0/2756155508 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe8e410f200 0x7fe8e411d260 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:23.948 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.945+0000 7fe8eb5206c0 1 -- 10.20.193.27:0/2756155508 >> 10.20.193.27:0/2756155508 conn(0x7fe8e40ecb00 msgr2=0x7fe8e410cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:23.949 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.945+0000 7fe8eb5206c0 1 -- 10.20.193.27:0/2756155508 shutdown_connections 2026-02-13T02:24:23.949 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:23.945+0000 7fe8eb5206c0 1 -- 10.20.193.27:0/2756155508 wait complete. 2026-02-13T02:24:24.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:24 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2756155508' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:23 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2756155508' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:24.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:24 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2756155508' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:26.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:26 trial107 ceph-mon[27422]: pgmap v252: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 2 op/s 2026-02-13T02:24:26.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:25 trial027 ceph-mon[24567]: pgmap v252: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 2 op/s 2026-02-13T02:24:26.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:26 trial028 ceph-mon[26701]: pgmap v252: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.0 KiB/s rd, 2 op/s 2026-02-13T02:24:26.988 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:24:27.110 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:24:27.228 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.223+0000 7f8ac71f76c0 1 -- 10.20.193.27:0/1715122244 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8ac010f200 msgr2=0x7f8ac010f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:27.228 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.223+0000 7f8ac71f76c0 1 --2- 10.20.193.27:0/1715122244 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8ac010f200 0x7f8ac010f5d0 secure :-1 s=READY pgs=402 gs=1 cs=0 l=1 c_cookie=12269230506901230144 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8ab00099a0 tx=0x7f8ab002ed20 comp rx=0 tx=0).stop 2026-02-13T02:24:27.229 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.223+0000 7f8ac71f76c0 1 -- 10.20.193.27:0/1715122244 shutdown_connections 2026-02-13T02:24:27.229 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.223+0000 7f8ac71f76c0 1 --2- 10.20.193.27:0/1715122244 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8ac0115410 0x7f8ac0175db0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:27.229 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.223+0000 7f8ac71f76c0 1 --2- 10.20.193.27:0/1715122244 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8ac010fbf0 0x7f8ac0114c30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:27.229 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.223+0000 7f8ac71f76c0 1 --2- 10.20.193.27:0/1715122244 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8ac010f200 0x7f8ac010f5d0 unknown :-1 s=CLOSED pgs=402 gs=1 cs=0 l=1 c_cookie=12269230506901230144 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:27.229 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.223+0000 7f8ac71f76c0 1 -- 10.20.193.27:0/1715122244 >> 10.20.193.27:0/1715122244 conn(0x7f8ac00ecb00 msgr2=0x7f8ac010cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:27.229 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.223+0000 7f8ac71f76c0 1 -- 10.20.193.27:0/1715122244 shutdown_connections 2026-02-13T02:24:27.229 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.223+0000 7f8ac71f76c0 1 -- 10.20.193.27:0/1715122244 wait complete. 2026-02-13T02:24:27.229 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.224+0000 7f8ac71f76c0 1 Processor -- start 2026-02-13T02:24:27.229 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.224+0000 7f8ac71f76c0 1 -- start start 2026-02-13T02:24:27.229 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.224+0000 7f8ac71f76c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8ac010f200 0x7f8ac011d350 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:27.229 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.224+0000 7f8ac71f76c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8ac0115410 0x7f8ac011d890 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:27.230 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.224+0000 7f8ac71f76c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8ac0122c50 0x7f8ac01230f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:27.230 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.224+0000 7f8ac71f76c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac0123630 con 0x7f8ac010f200 2026-02-13T02:24:27.230 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.224+0000 7f8ac71f76c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac01237a0 con 0x7f8ac0122c50 2026-02-13T02:24:27.230 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.224+0000 7f8ac71f76c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac0123960 con 0x7f8ac0115410 2026-02-13T02:24:27.230 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.224+0000 7f8ac4f686c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8ac010f200 0x7f8ac011d350 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:27.230 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.224+0000 7f8ac4f686c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8ac010f200 0x7f8ac011d350 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:41496/0 (socket says 10.20.193.27:41496) 2026-02-13T02:24:27.230 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.224+0000 7f8ac4f686c0 1 -- 10.20.193.27:0/435968997 learned_addr learned my addr 10.20.193.27:0/435968997 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:24:27.230 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.224+0000 7f8ac57696c0 1 --2- 10.20.193.27:0/435968997 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8ac0122c50 0x7f8ac01230f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:27.230 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.224+0000 7f8ab7fff6c0 1 --2- 10.20.193.27:0/435968997 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8ac0115410 0x7f8ac011d890 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:27.230 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.224+0000 7f8ac4f686c0 1 -- 10.20.193.27:0/435968997 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8ac0115410 msgr2=0x7f8ac011d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:27.230 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.224+0000 7f8ac4f686c0 1 --2- 10.20.193.27:0/435968997 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8ac0115410 0x7f8ac011d890 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:27.230 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.224+0000 7f8ac4f686c0 1 -- 10.20.193.27:0/435968997 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8ac0122c50 msgr2=0x7f8ac01230f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:27.231 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.224+0000 7f8ac4f686c0 1 --2- 10.20.193.27:0/435968997 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8ac0122c50 0x7f8ac01230f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:27.231 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.224+0000 7f8ac4f686c0 1 -- 10.20.193.27:0/435968997 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab0009590 con 0x7f8ac010f200 2026-02-13T02:24:27.231 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.224+0000 7f8ac4f686c0 1 --2- 10.20.193.27:0/435968997 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8ac010f200 0x7f8ac011d350 secure :-1 s=READY pgs=403 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8ab002f230 tx=0x7f8ab002fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:27.231 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.224+0000 7f8ab5ffb6c0 1 -- 10.20.193.27:0/435968997 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8ab0002ba0 con 0x7f8ac010f200 2026-02-13T02:24:27.231 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.225+0000 7f8ab5ffb6c0 1 -- 10.20.193.27:0/435968997 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f8ab0002d00 con 0x7f8ac010f200 2026-02-13T02:24:27.231 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.225+0000 7f8ac71f76c0 1 -- 10.20.193.27:0/435968997 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ac0163de0 con 0x7f8ac010f200 2026-02-13T02:24:27.231 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.225+0000 7f8ab5ffb6c0 1 -- 10.20.193.27:0/435968997 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8ab00372f0 con 0x7f8ac010f200 2026-02-13T02:24:27.231 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.225+0000 7f8ac71f76c0 1 -- 10.20.193.27:0/435968997 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ac0164260 con 0x7f8ac010f200 2026-02-13T02:24:27.231 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.225+0000 7f8ac71f76c0 1 -- 10.20.193.27:0/435968997 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ac01104d0 con 0x7f8ac010f200 2026-02-13T02:24:27.231 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.226+0000 7f8ab5ffb6c0 1 -- 10.20.193.27:0/435968997 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f8ab00078d0 con 0x7f8ac010f200 2026-02-13T02:24:27.232 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.227+0000 7f8ab5ffb6c0 1 --2- 10.20.193.27:0/435968997 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8a9c0b5600 0x7f8a9c0d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:27.232 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.227+0000 7f8ab5ffb6c0 1 -- 10.20.193.27:0/435968997 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f8ab0107b90 con 0x7f8ac010f200 2026-02-13T02:24:27.232 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.227+0000 7f8ab7fff6c0 1 --2- 10.20.193.27:0/435968997 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8a9c0b5600 0x7f8a9c0d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:27.232 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.227+0000 7f8ab5ffb6c0 1 -- 10.20.193.27:0/435968997 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f8ab00ca6a0 con 0x7f8ac010f200 2026-02-13T02:24:27.233 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.227+0000 7f8ab7fff6c0 1 --2- 10.20.193.27:0/435968997 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8a9c0b5600 0x7f8a9c0d5ab0 secure :-1 s=READY pgs=265 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8ac0123ce0 tx=0x7f8aa8005e60 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:27.342 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.339+0000 7f8ac71f76c0 1 -- 10.20.193.27:0/435968997 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8ac010f5d0 con 0x7f8ac010f200 2026-02-13T02:24:27.342 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.339+0000 7f8ab5ffb6c0 1 -- 10.20.193.27:0/435968997 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f8ab00cd7c0 con 0x7f8ac010f200 2026-02-13T02:24:27.342 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:24:27.342 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:24:27.343 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.340+0000 7f8ac71f76c0 1 -- 10.20.193.27:0/435968997 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8a9c0b5600 msgr2=0x7f8a9c0d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:27.343 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.340+0000 7f8ac71f76c0 1 --2- 10.20.193.27:0/435968997 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8a9c0b5600 0x7f8a9c0d5ab0 secure :-1 s=READY pgs=265 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8ac0123ce0 tx=0x7f8aa8005e60 comp rx=0 tx=0).stop 2026-02-13T02:24:27.344 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.340+0000 7f8ac71f76c0 1 -- 10.20.193.27:0/435968997 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8ac010f200 msgr2=0x7f8ac011d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:27.344 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.340+0000 7f8ac71f76c0 1 --2- 10.20.193.27:0/435968997 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8ac010f200 0x7f8ac011d350 secure :-1 s=READY pgs=403 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8ab002f230 tx=0x7f8ab002fd40 comp rx=0 tx=0).stop 2026-02-13T02:24:27.344 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.340+0000 7f8ac71f76c0 1 -- 10.20.193.27:0/435968997 shutdown_connections 2026-02-13T02:24:27.344 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.340+0000 7f8ac71f76c0 1 --2- 10.20.193.27:0/435968997 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8a9c0b5600 0x7f8a9c0d5ab0 unknown :-1 s=CLOSED pgs=265 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:27.344 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.340+0000 7f8ac71f76c0 1 --2- 10.20.193.27:0/435968997 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8ac0122c50 0x7f8ac01230f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:27.344 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.340+0000 7f8ac71f76c0 1 --2- 10.20.193.27:0/435968997 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8ac0115410 0x7f8ac011d890 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:27.344 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.340+0000 7f8ac71f76c0 1 --2- 10.20.193.27:0/435968997 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8ac010f200 0x7f8ac011d350 unknown :-1 s=CLOSED pgs=403 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:27.344 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.340+0000 7f8ac71f76c0 1 -- 10.20.193.27:0/435968997 >> 10.20.193.27:0/435968997 conn(0x7f8ac00ecb00 msgr2=0x7f8ac010cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:27.344 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.340+0000 7f8ac71f76c0 1 -- 10.20.193.27:0/435968997 shutdown_connections 2026-02-13T02:24:27.344 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:27.340+0000 7f8ac71f76c0 1 -- 10.20.193.27:0/435968997 wait complete. 2026-02-13T02:24:28.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:28 trial107 ceph-mon[27422]: pgmap v253: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 522 B/s rd, 1 op/s 2026-02-13T02:24:28.251 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:28 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/435968997' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:28.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:27 trial027 ceph-mon[24567]: pgmap v253: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 522 B/s rd, 1 op/s 2026-02-13T02:24:28.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:27 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/435968997' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:28.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:28 trial028 ceph-mon[26701]: pgmap v253: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 522 B/s rd, 1 op/s 2026-02-13T02:24:28.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:28 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/435968997' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:30.384 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:24:30.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:30 trial027 ceph-mon[24567]: pgmap v254: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 965 B/s rd, 1 op/s 2026-02-13T02:24:30.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:30 trial028 ceph-mon[26701]: pgmap v254: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 965 B/s rd, 1 op/s 2026-02-13T02:24:30.500 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:24:30.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:30 trial107 ceph-mon[27422]: pgmap v254: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 965 B/s rd, 1 op/s 2026-02-13T02:24:30.623 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.619+0000 7f9a5dd786c0 1 -- 10.20.193.27:0/2646090043 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9a58115440 msgr2=0x7f9a58175de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:30.623 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.619+0000 7f9a5dd786c0 1 --2- 10.20.193.27:0/2646090043 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9a58115440 0x7f9a58175de0 secure :-1 s=READY pgs=404 gs=1 cs=0 l=1 c_cookie=11409910938643721327 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a4c00aa70 tx=0x7f9a4c02fde0 comp rx=0 tx=0).stop 2026-02-13T02:24:30.623 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.619+0000 7f9a5dd786c0 1 -- 10.20.193.27:0/2646090043 shutdown_connections 2026-02-13T02:24:30.624 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.619+0000 7f9a5dd786c0 1 --2- 10.20.193.27:0/2646090043 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9a58115440 0x7f9a58175de0 unknown :-1 s=CLOSED pgs=404 gs=1 cs=0 l=1 c_cookie=11409910938643721327 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:30.624 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.619+0000 7f9a5dd786c0 1 --2- 10.20.193.27:0/2646090043 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a5810fc20 0x7f9a58114c60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:30.624 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.619+0000 7f9a5dd786c0 1 --2- 10.20.193.27:0/2646090043 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9a5810f230 0x7f9a5810f600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:30.624 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.619+0000 7f9a5dd786c0 1 -- 10.20.193.27:0/2646090043 >> 10.20.193.27:0/2646090043 conn(0x7f9a580ecb30 msgr2=0x7f9a5810cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:30.624 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.619+0000 7f9a5dd786c0 1 -- 10.20.193.27:0/2646090043 shutdown_connections 2026-02-13T02:24:30.624 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.619+0000 7f9a5dd786c0 1 -- 10.20.193.27:0/2646090043 wait complete. 2026-02-13T02:24:30.624 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.620+0000 7f9a5dd786c0 1 Processor -- start 2026-02-13T02:24:30.624 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.620+0000 7f9a5dd786c0 1 -- start start 2026-02-13T02:24:30.624 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.620+0000 7f9a5dd786c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9a5810f230 0x7f9a5811d200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:30.624 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.620+0000 7f9a5dd786c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9a58115440 0x7f9a5811d740 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:30.624 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.620+0000 7f9a5dd786c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a58122b00 0x7f9a58122fa0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:30.625 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.620+0000 7f9a5dd786c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9a58123510 con 0x7f9a5810f230 2026-02-13T02:24:30.625 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.620+0000 7f9a5dd786c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9a58123680 con 0x7f9a58122b00 2026-02-13T02:24:30.625 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.620+0000 7f9a5dd786c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9a58123840 con 0x7f9a58115440 2026-02-13T02:24:30.625 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.620+0000 7f9a577fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9a5810f230 0x7f9a5811d200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:30.625 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.620+0000 7f9a577fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9a5810f230 0x7f9a5811d200 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:41522/0 (socket says 10.20.193.27:41522) 2026-02-13T02:24:30.625 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.620+0000 7f9a577fe6c0 1 -- 10.20.193.27:0/2022996399 learned_addr learned my addr 10.20.193.27:0/2022996399 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:24:30.625 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.620+0000 7f9a56ffd6c0 1 --2- 10.20.193.27:0/2022996399 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9a58115440 0x7f9a5811d740 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:30.625 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.620+0000 7f9a57fff6c0 1 --2- 10.20.193.27:0/2022996399 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a58122b00 0x7f9a58122fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:30.625 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.620+0000 7f9a577fe6c0 1 -- 10.20.193.27:0/2022996399 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9a58115440 msgr2=0x7f9a5811d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:30.625 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.620+0000 7f9a577fe6c0 1 --2- 10.20.193.27:0/2022996399 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9a58115440 0x7f9a5811d740 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:30.625 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.620+0000 7f9a577fe6c0 1 -- 10.20.193.27:0/2022996399 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a58122b00 msgr2=0x7f9a58122fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:30.626 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.620+0000 7f9a577fe6c0 1 --2- 10.20.193.27:0/2022996399 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a58122b00 0x7f9a58122fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:30.626 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.620+0000 7f9a577fe6c0 1 -- 10.20.193.27:0/2022996399 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a4c00a660 con 0x7f9a5810f230 2026-02-13T02:24:30.626 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.620+0000 7f9a56ffd6c0 1 --2- 10.20.193.27:0/2022996399 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9a58115440 0x7f9a5811d740 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:24:30.626 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.620+0000 7f9a577fe6c0 1 --2- 10.20.193.27:0/2022996399 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9a5810f230 0x7f9a5811d200 secure :-1 s=READY pgs=405 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a4400b6c0 tx=0x7f9a4400bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:30.626 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.620+0000 7f9a54ff96c0 1 -- 10.20.193.27:0/2022996399 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9a440044a0 con 0x7f9a5810f230 2026-02-13T02:24:30.626 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.620+0000 7f9a5dd786c0 1 -- 10.20.193.27:0/2022996399 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a58163cf0 con 0x7f9a5810f230 2026-02-13T02:24:30.626 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.620+0000 7f9a54ff96c0 1 -- 10.20.193.27:0/2022996399 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9a44009030 con 0x7f9a5810f230 2026-02-13T02:24:30.626 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.620+0000 7f9a54ff96c0 1 -- 10.20.193.27:0/2022996399 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9a4400cbe0 con 0x7f9a5810f230 2026-02-13T02:24:30.626 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.620+0000 7f9a5dd786c0 1 -- 10.20.193.27:0/2022996399 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a58164180 con 0x7f9a5810f230 2026-02-13T02:24:30.626 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.621+0000 7f9a5dd786c0 1 -- 10.20.193.27:0/2022996399 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a58110500 con 0x7f9a5810f230 2026-02-13T02:24:30.627 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.622+0000 7f9a54ff96c0 1 -- 10.20.193.27:0/2022996399 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f9a440026b0 con 0x7f9a5810f230 2026-02-13T02:24:30.627 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.623+0000 7f9a54ff96c0 1 --2- 10.20.193.27:0/2022996399 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9a2c0b5680 0x7f9a2c0d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:30.627 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.623+0000 7f9a56ffd6c0 1 --2- 10.20.193.27:0/2022996399 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9a2c0b5680 0x7f9a2c0d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:30.628 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.623+0000 7f9a54ff96c0 1 -- 10.20.193.27:0/2022996399 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f9a440e3960 con 0x7f9a5810f230 2026-02-13T02:24:30.628 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.623+0000 7f9a54ff96c0 1 -- 10.20.193.27:0/2022996399 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f9a440e3de0 con 0x7f9a5810f230 2026-02-13T02:24:30.628 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.624+0000 7f9a56ffd6c0 1 --2- 10.20.193.27:0/2022996399 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9a2c0b5680 0x7f9a2c0d5b30 secure :-1 s=READY pgs=266 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a58123b90 tx=0x7f9a48005f10 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:30.739 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.738+0000 7f9a5dd786c0 1 -- 10.20.193.27:0/2022996399 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f9a5810fc20 con 0x7f9a5810f230 2026-02-13T02:24:30.739 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.738+0000 7f9a54ff96c0 1 -- 10.20.193.27:0/2022996399 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f9a440a63e0 con 0x7f9a5810f230 2026-02-13T02:24:30.740 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:24:30.740 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:24:30.741 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.739+0000 7f9a5dd786c0 1 -- 10.20.193.27:0/2022996399 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9a2c0b5680 msgr2=0x7f9a2c0d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:30.741 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.739+0000 7f9a5dd786c0 1 --2- 10.20.193.27:0/2022996399 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9a2c0b5680 0x7f9a2c0d5b30 secure :-1 s=READY pgs=266 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a58123b90 tx=0x7f9a48005f10 comp rx=0 tx=0).stop 2026-02-13T02:24:30.741 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.739+0000 7f9a5dd786c0 1 -- 10.20.193.27:0/2022996399 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9a5810f230 msgr2=0x7f9a5811d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:30.741 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.739+0000 7f9a5dd786c0 1 --2- 10.20.193.27:0/2022996399 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9a5810f230 0x7f9a5811d200 secure :-1 s=READY pgs=405 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a4400b6c0 tx=0x7f9a4400bb90 comp rx=0 tx=0).stop 2026-02-13T02:24:30.741 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.739+0000 7f9a5dd786c0 1 -- 10.20.193.27:0/2022996399 shutdown_connections 2026-02-13T02:24:30.741 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.739+0000 7f9a5dd786c0 1 --2- 10.20.193.27:0/2022996399 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9a2c0b5680 0x7f9a2c0d5b30 unknown :-1 s=CLOSED pgs=266 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:30.741 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.739+0000 7f9a5dd786c0 1 --2- 10.20.193.27:0/2022996399 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a58122b00 0x7f9a58122fa0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:30.741 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.739+0000 7f9a5dd786c0 1 --2- 10.20.193.27:0/2022996399 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9a58115440 0x7f9a5811d740 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:30.742 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.739+0000 7f9a5dd786c0 1 --2- 10.20.193.27:0/2022996399 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9a5810f230 0x7f9a5811d200 unknown :-1 s=CLOSED pgs=405 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:30.742 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.739+0000 7f9a5dd786c0 1 -- 10.20.193.27:0/2022996399 >> 10.20.193.27:0/2022996399 conn(0x7f9a580ecb30 msgr2=0x7f9a5810cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:30.742 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.739+0000 7f9a5dd786c0 1 -- 10.20.193.27:0/2022996399 shutdown_connections 2026-02-13T02:24:30.742 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:30.739+0000 7f9a5dd786c0 1 -- 10.20.193.27:0/2022996399 wait complete. 2026-02-13T02:24:31.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:31 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2022996399' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:31.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:31 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2022996399' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:31.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:31 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2022996399' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:32.394 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:32 trial028 ceph-mon[26701]: pgmap v255: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:32.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:32 trial027 ceph-mon[24567]: pgmap v255: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:32.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:32 trial107 ceph-mon[27422]: pgmap v255: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:33.782 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:24:33.904 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:24:34.023 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.020+0000 7fd96f61f6c0 1 -- 10.20.193.27:0/48418876 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd96810f200 msgr2=0x7fd96810f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:34.023 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.020+0000 7fd96f61f6c0 1 --2- 10.20.193.27:0/48418876 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd96810f200 0x7fd96810f5d0 secure :-1 s=READY pgs=406 gs=1 cs=0 l=1 c_cookie=12124158398730406762 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd9580099a0 tx=0x7fd95802ed20 comp rx=0 tx=0).stop 2026-02-13T02:24:34.023 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.020+0000 7fd96f61f6c0 1 -- 10.20.193.27:0/48418876 shutdown_connections 2026-02-13T02:24:34.024 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.020+0000 7fd96f61f6c0 1 --2- 10.20.193.27:0/48418876 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd968115410 0x7fd968175db0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:34.024 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.020+0000 7fd96f61f6c0 1 --2- 10.20.193.27:0/48418876 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd96810fbf0 0x7fd968114c30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:34.024 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.020+0000 7fd96f61f6c0 1 --2- 10.20.193.27:0/48418876 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd96810f200 0x7fd96810f5d0 unknown :-1 s=CLOSED pgs=406 gs=1 cs=0 l=1 c_cookie=12124158398730406762 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:34.024 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.020+0000 7fd96f61f6c0 1 -- 10.20.193.27:0/48418876 >> 10.20.193.27:0/48418876 conn(0x7fd9680ecb00 msgr2=0x7fd96810cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:34.024 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.020+0000 7fd96f61f6c0 1 -- 10.20.193.27:0/48418876 shutdown_connections 2026-02-13T02:24:34.024 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.020+0000 7fd96f61f6c0 1 -- 10.20.193.27:0/48418876 wait complete. 2026-02-13T02:24:34.024 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.020+0000 7fd96f61f6c0 1 Processor -- start 2026-02-13T02:24:34.024 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.020+0000 7fd96f61f6c0 1 -- start start 2026-02-13T02:24:34.024 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.021+0000 7fd96f61f6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd96810f200 0x7fd96811d350 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:34.024 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.021+0000 7fd96f61f6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd968115410 0x7fd96811d890 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:34.024 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.021+0000 7fd96f61f6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd968122c50 0x7fd9681230f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:34.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.021+0000 7fd96f61f6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd968123660 con 0x7fd96810f200 2026-02-13T02:24:34.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.021+0000 7fd96f61f6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd9681237d0 con 0x7fd968115410 2026-02-13T02:24:34.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.021+0000 7fd96f61f6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd968123990 con 0x7fd968122c50 2026-02-13T02:24:34.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.021+0000 7fd96d3906c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd96810f200 0x7fd96811d350 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:34.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.021+0000 7fd96d3906c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd96810f200 0x7fd96811d350 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:41540/0 (socket says 10.20.193.27:41540) 2026-02-13T02:24:34.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.021+0000 7fd96d3906c0 1 -- 10.20.193.27:0/1861158439 learned_addr learned my addr 10.20.193.27:0/1861158439 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:24:34.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.021+0000 7fd96db916c0 1 --2- 10.20.193.27:0/1861158439 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd968122c50 0x7fd9681230f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:34.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.021+0000 7fd96cb8f6c0 1 --2- 10.20.193.27:0/1861158439 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd968115410 0x7fd96811d890 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:34.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.021+0000 7fd96d3906c0 1 -- 10.20.193.27:0/1861158439 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd968122c50 msgr2=0x7fd9681230f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:34.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.021+0000 7fd96d3906c0 1 --2- 10.20.193.27:0/1861158439 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd968122c50 0x7fd9681230f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:34.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.021+0000 7fd96d3906c0 1 -- 10.20.193.27:0/1861158439 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd968115410 msgr2=0x7fd96811d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:34.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.021+0000 7fd96d3906c0 1 --2- 10.20.193.27:0/1861158439 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd968115410 0x7fd96811d890 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:34.026 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.021+0000 7fd96d3906c0 1 -- 10.20.193.27:0/1861158439 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd958009590 con 0x7fd96810f200 2026-02-13T02:24:34.026 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.021+0000 7fd96d3906c0 1 --2- 10.20.193.27:0/1861158439 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd96810f200 0x7fd96811d350 secure :-1 s=READY pgs=407 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd95802f230 tx=0x7fd95802fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:34.026 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.021+0000 7fd9567fc6c0 1 -- 10.20.193.27:0/1861158439 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd958002ba0 con 0x7fd96810f200 2026-02-13T02:24:34.026 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.021+0000 7fd96f61f6c0 1 -- 10.20.193.27:0/1861158439 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd968163de0 con 0x7fd96810f200 2026-02-13T02:24:34.026 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.021+0000 7fd9567fc6c0 1 -- 10.20.193.27:0/1861158439 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd958002d00 con 0x7fd96810f200 2026-02-13T02:24:34.026 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.021+0000 7fd9567fc6c0 1 -- 10.20.193.27:0/1861158439 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd9580372f0 con 0x7fd96810f200 2026-02-13T02:24:34.026 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.021+0000 7fd96f61f6c0 1 -- 10.20.193.27:0/1861158439 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd968164210 con 0x7fd96810f200 2026-02-13T02:24:34.026 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.022+0000 7fd96f61f6c0 1 -- 10.20.193.27:0/1861158439 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9681104d0 con 0x7fd96810f200 2026-02-13T02:24:34.027 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.023+0000 7fd9567fc6c0 1 -- 10.20.193.27:0/1861158439 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7fd9580078d0 con 0x7fd96810f200 2026-02-13T02:24:34.027 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.024+0000 7fd9567fc6c0 1 --2- 10.20.193.27:0/1861158439 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd9440b5600 0x7fd9440d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:34.027 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.024+0000 7fd96cb8f6c0 1 --2- 10.20.193.27:0/1861158439 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd9440b5600 0x7fd9440d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:34.027 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.024+0000 7fd9567fc6c0 1 -- 10.20.193.27:0/1861158439 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7fd958107be0 con 0x7fd96810f200 2026-02-13T02:24:34.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.024+0000 7fd96cb8f6c0 1 --2- 10.20.193.27:0/1861158439 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd9440b5600 0x7fd9440d5ab0 secure :-1 s=READY pgs=267 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd95c005980 tx=0x7fd95c005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:34.028 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.024+0000 7fd9567fc6c0 1 -- 10.20.193.27:0/1861158439 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fd9580ce100 con 0x7fd96810f200 2026-02-13T02:24:34.138 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.137+0000 7fd96f61f6c0 1 -- 10.20.193.27:0/1861158439 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd96810f5d0 con 0x7fd96810f200 2026-02-13T02:24:34.139 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:34 trial027 ceph-mon[24567]: pgmap v256: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:24:34.139 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.137+0000 7fd9567fc6c0 1 -- 10.20.193.27:0/1861158439 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7fd9580cd850 con 0x7fd96810f200 2026-02-13T02:24:34.139 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:24:34.140 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:24:34.140 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.138+0000 7fd96f61f6c0 1 -- 10.20.193.27:0/1861158439 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd9440b5600 msgr2=0x7fd9440d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:34.141 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.138+0000 7fd96f61f6c0 1 --2- 10.20.193.27:0/1861158439 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd9440b5600 0x7fd9440d5ab0 secure :-1 s=READY pgs=267 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd95c005980 tx=0x7fd95c005910 comp rx=0 tx=0).stop 2026-02-13T02:24:34.141 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.138+0000 7fd96f61f6c0 1 -- 10.20.193.27:0/1861158439 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd96810f200 msgr2=0x7fd96811d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:34.141 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.138+0000 7fd96f61f6c0 1 --2- 10.20.193.27:0/1861158439 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd96810f200 0x7fd96811d350 secure :-1 s=READY pgs=407 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd95802f230 tx=0x7fd95802fd40 comp rx=0 tx=0).stop 2026-02-13T02:24:34.141 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.138+0000 7fd96f61f6c0 1 -- 10.20.193.27:0/1861158439 shutdown_connections 2026-02-13T02:24:34.141 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.138+0000 7fd96f61f6c0 1 --2- 10.20.193.27:0/1861158439 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd9440b5600 0x7fd9440d5ab0 unknown :-1 s=CLOSED pgs=267 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:34.141 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.138+0000 7fd96f61f6c0 1 --2- 10.20.193.27:0/1861158439 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd968122c50 0x7fd9681230f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:34.141 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.138+0000 7fd96f61f6c0 1 --2- 10.20.193.27:0/1861158439 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd968115410 0x7fd96811d890 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:34.141 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.138+0000 7fd96f61f6c0 1 --2- 10.20.193.27:0/1861158439 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd96810f200 0x7fd96811d350 unknown :-1 s=CLOSED pgs=407 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:34.141 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.138+0000 7fd96f61f6c0 1 -- 10.20.193.27:0/1861158439 >> 10.20.193.27:0/1861158439 conn(0x7fd9680ecb00 msgr2=0x7fd96810cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:34.141 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.138+0000 7fd96f61f6c0 1 -- 10.20.193.27:0/1861158439 shutdown_connections 2026-02-13T02:24:34.141 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:34.138+0000 7fd96f61f6c0 1 -- 10.20.193.27:0/1861158439 wait complete. 2026-02-13T02:24:34.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:34 trial028 ceph-mon[26701]: pgmap v256: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:24:34.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:34 trial107 ceph-mon[27422]: pgmap v256: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:24:35.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:35 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1861158439' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:35.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:35 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1861158439' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:35.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:35 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1861158439' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:36.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:36 trial027 ceph-mon[24567]: pgmap v257: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:36.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:36 trial028 ceph-mon[26701]: pgmap v257: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:36.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:36 trial107 ceph-mon[27422]: pgmap v257: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:37.184 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:24:37.302 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:24:37.422 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.418+0000 7f2bccb416c0 1 -- 10.20.193.27:0/3503321643 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2bc810fbf0 msgr2=0x7f2bc8114c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:37.422 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.418+0000 7f2bccb416c0 1 --2- 10.20.193.27:0/3503321643 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2bc810fbf0 0x7f2bc8114c30 secure :-1 s=READY pgs=408 gs=1 cs=0 l=1 c_cookie=11115243493637662639 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2bb00099a0 tx=0x7f2bb002ed20 comp rx=0 tx=0).stop 2026-02-13T02:24:37.422 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.418+0000 7f2bccb416c0 1 -- 10.20.193.27:0/3503321643 shutdown_connections 2026-02-13T02:24:37.422 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.418+0000 7f2bccb416c0 1 --2- 10.20.193.27:0/3503321643 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2bc8115410 0x7f2bc8175db0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:37.422 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.418+0000 7f2bccb416c0 1 --2- 10.20.193.27:0/3503321643 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2bc810fbf0 0x7f2bc8114c30 unknown :-1 s=CLOSED pgs=408 gs=1 cs=0 l=1 c_cookie=11115243493637662639 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:37.423 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.418+0000 7f2bccb416c0 1 --2- 10.20.193.27:0/3503321643 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2bc810f200 0x7f2bc810f5d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:37.423 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.418+0000 7f2bccb416c0 1 -- 10.20.193.27:0/3503321643 >> 10.20.193.27:0/3503321643 conn(0x7f2bc80ecb00 msgr2=0x7f2bc810cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:37.423 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.418+0000 7f2bccb416c0 1 -- 10.20.193.27:0/3503321643 shutdown_connections 2026-02-13T02:24:37.423 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.418+0000 7f2bccb416c0 1 -- 10.20.193.27:0/3503321643 wait complete. 2026-02-13T02:24:37.423 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.419+0000 7f2bccb416c0 1 Processor -- start 2026-02-13T02:24:37.423 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.419+0000 7f2bccb416c0 1 -- start start 2026-02-13T02:24:37.423 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.419+0000 7f2bccb416c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2bc810f200 0x7f2bc80f4c80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:37.423 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.419+0000 7f2bccb416c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2bc8115410 0x7f2bc8175cb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:37.423 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.419+0000 7f2bccb416c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2bc80f52b0 0x7f2bc80fa580 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:37.423 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.419+0000 7f2bccb416c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc80f59a0 con 0x7f2bc810f200 2026-02-13T02:24:37.423 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.419+0000 7f2bccb416c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc80faac0 con 0x7f2bc80f52b0 2026-02-13T02:24:37.424 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.419+0000 7f2bccb416c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc80fac80 con 0x7f2bc8115410 2026-02-13T02:24:37.424 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.419+0000 7f2bc65756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2bc810f200 0x7f2bc80f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:37.424 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.419+0000 7f2bc65756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2bc810f200 0x7f2bc80f4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:56568/0 (socket says 10.20.193.27:56568) 2026-02-13T02:24:37.424 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.419+0000 7f2bc65756c0 1 -- 10.20.193.27:0/501166122 learned_addr learned my addr 10.20.193.27:0/501166122 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:24:37.424 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.419+0000 7f2bc6d766c0 1 --2- 10.20.193.27:0/501166122 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2bc80f52b0 0x7f2bc80fa580 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:37.424 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.419+0000 7f2bc5d746c0 1 --2- 10.20.193.27:0/501166122 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2bc8115410 0x7f2bc8175cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:37.424 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.419+0000 7f2bc65756c0 1 -- 10.20.193.27:0/501166122 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2bc8115410 msgr2=0x7f2bc8175cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:37.424 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.419+0000 7f2bc65756c0 1 --2- 10.20.193.27:0/501166122 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2bc8115410 0x7f2bc8175cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:37.424 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.419+0000 7f2bc65756c0 1 -- 10.20.193.27:0/501166122 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2bc80f52b0 msgr2=0x7f2bc80fa580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:37.424 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.419+0000 7f2bc65756c0 1 --2- 10.20.193.27:0/501166122 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2bc80f52b0 0x7f2bc80fa580 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:37.424 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.419+0000 7f2bc65756c0 1 -- 10.20.193.27:0/501166122 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb0009590 con 0x7f2bc810f200 2026-02-13T02:24:37.424 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.419+0000 7f2bc65756c0 1 --2- 10.20.193.27:0/501166122 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2bc810f200 0x7f2bc80f4c80 secure :-1 s=READY pgs=409 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2bbc0092e0 tx=0x7f2bbc00bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:37.425 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.419+0000 7f2baf7fe6c0 1 -- 10.20.193.27:0/501166122 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2bbc004250 con 0x7f2bc810f200 2026-02-13T02:24:37.425 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.419+0000 7f2baf7fe6c0 1 -- 10.20.193.27:0/501166122 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f2bbc0043b0 con 0x7f2bc810f200 2026-02-13T02:24:37.425 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.419+0000 7f2bccb416c0 1 -- 10.20.193.27:0/501166122 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bc80faf60 con 0x7f2bc810f200 2026-02-13T02:24:37.425 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.419+0000 7f2baf7fe6c0 1 -- 10.20.193.27:0/501166122 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2bbc00cd70 con 0x7f2bc810f200 2026-02-13T02:24:37.425 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.419+0000 7f2bccb416c0 1 -- 10.20.193.27:0/501166122 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bc80fb6f0 con 0x7f2bc810f200 2026-02-13T02:24:37.425 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.420+0000 7f2bccb416c0 1 -- 10.20.193.27:0/501166122 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bc81104d0 con 0x7f2bc810f200 2026-02-13T02:24:37.426 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.421+0000 7f2baf7fe6c0 1 -- 10.20.193.27:0/501166122 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f2bbc0026b0 con 0x7f2bc810f200 2026-02-13T02:24:37.426 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.422+0000 7f2baf7fe6c0 1 --2- 10.20.193.27:0/501166122 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f2ba00b5680 0x7f2ba00d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:37.426 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.422+0000 7f2bc5d746c0 1 --2- 10.20.193.27:0/501166122 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f2ba00b5680 0x7f2ba00d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:37.426 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.422+0000 7f2baf7fe6c0 1 -- 10.20.193.27:0/501166122 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f2bbc0e3940 con 0x7f2bc810f200 2026-02-13T02:24:37.426 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.422+0000 7f2baf7fe6c0 1 -- 10.20.193.27:0/501166122 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f2bbc0e3dc0 con 0x7f2bc810f200 2026-02-13T02:24:37.427 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.422+0000 7f2bc5d746c0 1 --2- 10.20.193.27:0/501166122 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f2ba00b5680 0x7f2ba00d5b30 secure :-1 s=READY pgs=268 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2bc80fb610 tx=0x7f2bb00023a0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:37.538 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.537+0000 7f2bccb416c0 1 -- 10.20.193.27:0/501166122 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2bc80f5dd0 con 0x7f2bc810f200 2026-02-13T02:24:37.538 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.537+0000 7f2baf7fe6c0 1 -- 10.20.193.27:0/501166122 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f2bbc0a63c0 con 0x7f2bc810f200 2026-02-13T02:24:37.538 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:24:37.539 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:24:37.540 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.538+0000 7f2bccb416c0 1 -- 10.20.193.27:0/501166122 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f2ba00b5680 msgr2=0x7f2ba00d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:37.540 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.538+0000 7f2bccb416c0 1 --2- 10.20.193.27:0/501166122 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f2ba00b5680 0x7f2ba00d5b30 secure :-1 s=READY pgs=268 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2bc80fb610 tx=0x7f2bb00023a0 comp rx=0 tx=0).stop 2026-02-13T02:24:37.540 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.538+0000 7f2bccb416c0 1 -- 10.20.193.27:0/501166122 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2bc810f200 msgr2=0x7f2bc80f4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:37.540 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.538+0000 7f2bccb416c0 1 --2- 10.20.193.27:0/501166122 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2bc810f200 0x7f2bc80f4c80 secure :-1 s=READY pgs=409 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2bbc0092e0 tx=0x7f2bbc00bc50 comp rx=0 tx=0).stop 2026-02-13T02:24:37.540 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.538+0000 7f2bccb416c0 1 -- 10.20.193.27:0/501166122 shutdown_connections 2026-02-13T02:24:37.540 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.538+0000 7f2bccb416c0 1 --2- 10.20.193.27:0/501166122 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f2ba00b5680 0x7f2ba00d5b30 unknown :-1 s=CLOSED pgs=268 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:37.540 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.538+0000 7f2bccb416c0 1 --2- 10.20.193.27:0/501166122 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2bc80f52b0 0x7f2bc80fa580 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:37.540 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.538+0000 7f2bccb416c0 1 --2- 10.20.193.27:0/501166122 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f2bc8115410 0x7f2bc8175cb0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:37.540 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.538+0000 7f2bccb416c0 1 --2- 10.20.193.27:0/501166122 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f2bc810f200 0x7f2bc80f4c80 unknown :-1 s=CLOSED pgs=409 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:37.540 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.538+0000 7f2bccb416c0 1 -- 10.20.193.27:0/501166122 >> 10.20.193.27:0/501166122 conn(0x7f2bc80ecb00 msgr2=0x7f2bc810d900 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:37.541 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.538+0000 7f2bccb416c0 1 -- 10.20.193.27:0/501166122 shutdown_connections 2026-02-13T02:24:37.541 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:37.538+0000 7f2bccb416c0 1 -- 10.20.193.27:0/501166122 wait complete. 2026-02-13T02:24:38.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:38 trial027 ceph-mon[24567]: pgmap v258: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:38.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:38 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/501166122' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:38.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:38 trial028 ceph-mon[26701]: pgmap v258: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:38.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:38 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/501166122' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:38.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:38 trial107 ceph-mon[27422]: pgmap v258: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:38.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:38 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/501166122' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:40.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:40 trial027 ceph-mon[24567]: pgmap v259: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:24:40.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:40 trial028 ceph-mon[26701]: pgmap v259: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:24:40.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:40 trial107 ceph-mon[27422]: pgmap v259: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:24:40.581 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:24:40.698 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:24:40.814 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.812+0000 7fc88b1e46c0 1 -- 10.20.193.27:0/2228389045 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc884115440 msgr2=0x7fc884175de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:40.814 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.812+0000 7fc88b1e46c0 1 --2- 10.20.193.27:0/2228389045 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc884115440 0x7fc884175de0 secure :-1 s=READY pgs=410 gs=1 cs=0 l=1 c_cookie=9533127330414567991 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc88000aa70 tx=0x7fc88002fde0 comp rx=0 tx=0).stop 2026-02-13T02:24:40.814 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.813+0000 7fc88b1e46c0 1 -- 10.20.193.27:0/2228389045 shutdown_connections 2026-02-13T02:24:40.814 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.813+0000 7fc88b1e46c0 1 --2- 10.20.193.27:0/2228389045 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc884115440 0x7fc884175de0 unknown :-1 s=CLOSED pgs=410 gs=1 cs=0 l=1 c_cookie=9533127330414567991 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:40.814 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.813+0000 7fc88b1e46c0 1 --2- 10.20.193.27:0/2228389045 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc88410fc20 0x7fc884114c60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:40.814 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.813+0000 7fc88b1e46c0 1 --2- 10.20.193.27:0/2228389045 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc88410f230 0x7fc88410f600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:40.814 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.813+0000 7fc88b1e46c0 1 -- 10.20.193.27:0/2228389045 >> 10.20.193.27:0/2228389045 conn(0x7fc8840ecb30 msgr2=0x7fc88410cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:40.814 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.813+0000 7fc88b1e46c0 1 -- 10.20.193.27:0/2228389045 shutdown_connections 2026-02-13T02:24:40.814 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.813+0000 7fc88b1e46c0 1 -- 10.20.193.27:0/2228389045 wait complete. 2026-02-13T02:24:40.814 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.813+0000 7fc88b1e46c0 1 Processor -- start 2026-02-13T02:24:40.815 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.813+0000 7fc88b1e46c0 1 -- start start 2026-02-13T02:24:40.815 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.813+0000 7fc88b1e46c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc88410f230 0x7fc88411d230 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:40.815 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.813+0000 7fc88b1e46c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc884115440 0x7fc88411d770 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:40.815 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.813+0000 7fc88b1e46c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc884122b30 0x7fc884122fd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:40.815 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.813+0000 7fc88b1e46c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc884123540 con 0x7fc88410f230 2026-02-13T02:24:40.815 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.813+0000 7fc88b1e46c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc8841236b0 con 0x7fc884115440 2026-02-13T02:24:40.815 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.813+0000 7fc88b1e46c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc884123870 con 0x7fc884122b30 2026-02-13T02:24:40.815 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.813+0000 7fc888f596c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc88410f230 0x7fc88411d230 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:40.815 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.813+0000 7fc888f596c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc88410f230 0x7fc88411d230 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:56590/0 (socket says 10.20.193.27:56590) 2026-02-13T02:24:40.815 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.813+0000 7fc888f596c0 1 -- 10.20.193.27:0/3842869299 learned_addr learned my addr 10.20.193.27:0/3842869299 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:24:40.815 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.813+0000 7fc888f596c0 1 -- 10.20.193.27:0/3842869299 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc884122b30 msgr2=0x7fc884122fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:40.815 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.813+0000 7fc888f596c0 1 --2- 10.20.193.27:0/3842869299 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc884122b30 0x7fc884122fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:40.815 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.813+0000 7fc888f596c0 1 -- 10.20.193.27:0/3842869299 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc884115440 msgr2=0x7fc88411d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:40.816 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.813+0000 7fc888f596c0 1 --2- 10.20.193.27:0/3842869299 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc884115440 0x7fc88411d770 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:40.816 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.813+0000 7fc888f596c0 1 -- 10.20.193.27:0/3842869299 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88000a660 con 0x7fc88410f230 2026-02-13T02:24:40.816 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.813+0000 7fc888f596c0 1 --2- 10.20.193.27:0/3842869299 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc88410f230 0x7fc88411d230 secure :-1 s=READY pgs=411 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc874005dc0 tx=0x7fc87400bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:40.816 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.814+0000 7fc879ffb6c0 1 -- 10.20.193.27:0/3842869299 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc874004250 con 0x7fc88410f230 2026-02-13T02:24:40.816 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.814+0000 7fc879ffb6c0 1 -- 10.20.193.27:0/3842869299 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc874009070 con 0x7fc88410f230 2026-02-13T02:24:40.816 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.814+0000 7fc88b1e46c0 1 -- 10.20.193.27:0/3842869299 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc884163d20 con 0x7fc88410f230 2026-02-13T02:24:40.816 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.814+0000 7fc879ffb6c0 1 -- 10.20.193.27:0/3842869299 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc87400cd70 con 0x7fc88410f230 2026-02-13T02:24:40.816 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.814+0000 7fc88b1e46c0 1 -- 10.20.193.27:0/3842869299 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8841641b0 con 0x7fc88410f230 2026-02-13T02:24:40.816 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.814+0000 7fc88b1e46c0 1 -- 10.20.193.27:0/3842869299 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc884110500 con 0x7fc88410f230 2026-02-13T02:24:40.816 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.815+0000 7fc879ffb6c0 1 -- 10.20.193.27:0/3842869299 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7fc8740028c0 con 0x7fc88410f230 2026-02-13T02:24:40.817 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.816+0000 7fc879ffb6c0 1 --2- 10.20.193.27:0/3842869299 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc8600b5600 0x7fc8600d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:40.817 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.816+0000 7fc879ffb6c0 1 -- 10.20.193.27:0/3842869299 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7fc8740e3b70 con 0x7fc88410f230 2026-02-13T02:24:40.817 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.816+0000 7fc87bfff6c0 1 --2- 10.20.193.27:0/3842869299 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc8600b5600 0x7fc8600d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:40.817 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.816+0000 7fc879ffb6c0 1 -- 10.20.193.27:0/3842869299 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fc8740a65f0 con 0x7fc88410f230 2026-02-13T02:24:40.817 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.816+0000 7fc87bfff6c0 1 --2- 10.20.193.27:0/3842869299 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc8600b5600 0x7fc8600d5ab0 secure :-1 s=READY pgs=269 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc86c005980 tx=0x7fc86c005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:40.933 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.931+0000 7fc88b1e46c0 1 -- 10.20.193.27:0/3842869299 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc88410f600 con 0x7fc88410f230 2026-02-13T02:24:40.933 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.931+0000 7fc879ffb6c0 1 -- 10.20.193.27:0/3842869299 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7fc8740a97a0 con 0x7fc88410f230 2026-02-13T02:24:40.933 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:24:40.933 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:24:40.934 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.932+0000 7fc88b1e46c0 1 -- 10.20.193.27:0/3842869299 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc8600b5600 msgr2=0x7fc8600d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:40.934 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.932+0000 7fc88b1e46c0 1 --2- 10.20.193.27:0/3842869299 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc8600b5600 0x7fc8600d5ab0 secure :-1 s=READY pgs=269 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc86c005980 tx=0x7fc86c005910 comp rx=0 tx=0).stop 2026-02-13T02:24:40.935 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.932+0000 7fc88b1e46c0 1 -- 10.20.193.27:0/3842869299 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc88410f230 msgr2=0x7fc88411d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:40.935 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.932+0000 7fc88b1e46c0 1 --2- 10.20.193.27:0/3842869299 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc88410f230 0x7fc88411d230 secure :-1 s=READY pgs=411 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc874005dc0 tx=0x7fc87400bbb0 comp rx=0 tx=0).stop 2026-02-13T02:24:40.935 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.932+0000 7fc88b1e46c0 1 -- 10.20.193.27:0/3842869299 shutdown_connections 2026-02-13T02:24:40.935 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.932+0000 7fc88b1e46c0 1 --2- 10.20.193.27:0/3842869299 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc8600b5600 0x7fc8600d5ab0 unknown :-1 s=CLOSED pgs=269 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:40.935 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.932+0000 7fc88b1e46c0 1 --2- 10.20.193.27:0/3842869299 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc884122b30 0x7fc884122fd0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:40.935 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.932+0000 7fc88b1e46c0 1 --2- 10.20.193.27:0/3842869299 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc884115440 0x7fc88411d770 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:40.935 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.932+0000 7fc88b1e46c0 1 --2- 10.20.193.27:0/3842869299 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc88410f230 0x7fc88411d230 unknown :-1 s=CLOSED pgs=411 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:40.935 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.932+0000 7fc88b1e46c0 1 -- 10.20.193.27:0/3842869299 >> 10.20.193.27:0/3842869299 conn(0x7fc8840ecb30 msgr2=0x7fc88410cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:40.935 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.932+0000 7fc88b1e46c0 1 -- 10.20.193.27:0/3842869299 shutdown_connections 2026-02-13T02:24:40.935 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:40.932+0000 7fc88b1e46c0 1 -- 10.20.193.27:0/3842869299 wait complete. 2026-02-13T02:24:41.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:41 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3842869299' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:41.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:41 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3842869299' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:41.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:41 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3842869299' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:42.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:42 trial027 ceph-mon[24567]: pgmap v260: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:42.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:42 trial028 ceph-mon[26701]: pgmap v260: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:42.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:42 trial107 ceph-mon[27422]: pgmap v260: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:43.974 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:24:44.093 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:24:44.214 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.209+0000 7f001d99f6c0 1 -- 10.20.193.27:0/1487975902 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f001810f200 msgr2=0x7f001810f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:44.215 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.209+0000 7f001d99f6c0 1 --2- 10.20.193.27:0/1487975902 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f001810f200 0x7f001810f5d0 secure :-1 s=READY pgs=412 gs=1 cs=0 l=1 c_cookie=14274361296497193733 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f000c0099a0 tx=0x7f000c02ed20 comp rx=0 tx=0).stop 2026-02-13T02:24:44.215 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.209+0000 7f001d99f6c0 1 -- 10.20.193.27:0/1487975902 shutdown_connections 2026-02-13T02:24:44.215 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.209+0000 7f001d99f6c0 1 --2- 10.20.193.27:0/1487975902 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f0018115410 0x7f0018175db0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:44.215 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.209+0000 7f001d99f6c0 1 --2- 10.20.193.27:0/1487975902 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f001810fbf0 0x7f0018114c30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:44.215 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.209+0000 7f001d99f6c0 1 --2- 10.20.193.27:0/1487975902 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f001810f200 0x7f001810f5d0 unknown :-1 s=CLOSED pgs=412 gs=1 cs=0 l=1 c_cookie=14274361296497193733 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:44.215 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.209+0000 7f001d99f6c0 1 -- 10.20.193.27:0/1487975902 >> 10.20.193.27:0/1487975902 conn(0x7f00180ecb00 msgr2=0x7f001810cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:44.215 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.209+0000 7f001d99f6c0 1 -- 10.20.193.27:0/1487975902 shutdown_connections 2026-02-13T02:24:44.216 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.209+0000 7f001d99f6c0 1 -- 10.20.193.27:0/1487975902 wait complete. 2026-02-13T02:24:44.216 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.210+0000 7f001d99f6c0 1 Processor -- start 2026-02-13T02:24:44.216 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.210+0000 7f001d99f6c0 1 -- start start 2026-02-13T02:24:44.216 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.210+0000 7f001d99f6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f001810f200 0x7f001811d300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:44.216 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.210+0000 7f001d99f6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0018115410 0x7f001811d840 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:44.216 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.210+0000 7f001d99f6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0018122c00 0x7f00181230a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:44.216 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.210+0000 7f001d99f6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0018123610 con 0x7f0018115410 2026-02-13T02:24:44.216 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.210+0000 7f001d99f6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0018123780 con 0x7f0018122c00 2026-02-13T02:24:44.216 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.210+0000 7f001d99f6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0018123940 con 0x7f001810f200 2026-02-13T02:24:44.217 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.210+0000 7f00167fc6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0018115410 0x7f001811d840 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:44.217 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.210+0000 7f00167fc6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0018115410 0x7f001811d840 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:56608/0 (socket says 10.20.193.27:56608) 2026-02-13T02:24:44.217 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.210+0000 7f00167fc6c0 1 -- 10.20.193.27:0/2546226949 learned_addr learned my addr 10.20.193.27:0/2546226949 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:24:44.217 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.210+0000 7f0016ffd6c0 1 --2- 10.20.193.27:0/2546226949 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f001810f200 0x7f001811d300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:44.217 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.210+0000 7f00167fc6c0 1 -- 10.20.193.27:0/2546226949 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f001810f200 msgr2=0x7f001811d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:44.217 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.210+0000 7f00167fc6c0 1 --2- 10.20.193.27:0/2546226949 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f001810f200 0x7f001811d300 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:44.217 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.210+0000 7f00177fe6c0 1 --2- 10.20.193.27:0/2546226949 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0018122c00 0x7f00181230a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:44.217 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.210+0000 7f00167fc6c0 1 -- 10.20.193.27:0/2546226949 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0018122c00 msgr2=0x7f00181230a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:44.217 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.210+0000 7f00167fc6c0 1 --2- 10.20.193.27:0/2546226949 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0018122c00 0x7f00181230a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:44.217 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.210+0000 7f00167fc6c0 1 -- 10.20.193.27:0/2546226949 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000c009590 con 0x7f0018115410 2026-02-13T02:24:44.217 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.210+0000 7f00177fe6c0 1 --2- 10.20.193.27:0/2546226949 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0018122c00 0x7f00181230a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:24:44.218 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.210+0000 7f00167fc6c0 1 --2- 10.20.193.27:0/2546226949 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0018115410 0x7f001811d840 secure :-1 s=READY pgs=413 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f000000b6c0 tx=0x7f000000bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:44.218 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.210+0000 7f001c9996c0 1 -- 10.20.193.27:0/2546226949 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f00000044a0 con 0x7f0018115410 2026-02-13T02:24:44.218 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.210+0000 7f001c9996c0 1 -- 10.20.193.27:0/2546226949 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f000000be50 con 0x7f0018115410 2026-02-13T02:24:44.218 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.210+0000 7f001d99f6c0 1 -- 10.20.193.27:0/2546226949 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0018163df0 con 0x7f0018115410 2026-02-13T02:24:44.218 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.211+0000 7f001c9996c0 1 -- 10.20.193.27:0/2546226949 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f000000cc90 con 0x7f0018115410 2026-02-13T02:24:44.218 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.211+0000 7f001d99f6c0 1 -- 10.20.193.27:0/2546226949 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0018164280 con 0x7f0018115410 2026-02-13T02:24:44.218 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.211+0000 7f001d99f6c0 1 -- 10.20.193.27:0/2546226949 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00181104d0 con 0x7f0018115410 2026-02-13T02:24:44.218 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.212+0000 7f001c9996c0 1 -- 10.20.193.27:0/2546226949 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f00000026b0 con 0x7f0018115410 2026-02-13T02:24:44.219 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.213+0000 7f001c9996c0 1 --2- 10.20.193.27:0/2546226949 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7effec0b5600 0x7effec0d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:44.219 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.213+0000 7f0016ffd6c0 1 --2- 10.20.193.27:0/2546226949 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7effec0b5600 0x7effec0d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:44.219 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.213+0000 7f001c9996c0 1 -- 10.20.193.27:0/2546226949 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f00000e3900 con 0x7f0018115410 2026-02-13T02:24:44.220 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.213+0000 7f0016ffd6c0 1 --2- 10.20.193.27:0/2546226949 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7effec0b5600 0x7effec0d5ab0 secure :-1 s=READY pgs=270 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f000c0023e0 tx=0x7f000c005ec0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:44.220 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.213+0000 7f001c9996c0 1 -- 10.20.193.27:0/2546226949 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f00000a9de0 con 0x7f0018115410 2026-02-13T02:24:44.333 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.331+0000 7f001d99f6c0 1 -- 10.20.193.27:0/2546226949 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f001810f5d0 con 0x7f0018115410 2026-02-13T02:24:44.333 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:44 trial027 ceph-mon[24567]: pgmap v261: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:24:44.334 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.331+0000 7f001c9996c0 1 -- 10.20.193.27:0/2546226949 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f00000a9530 con 0x7f0018115410 2026-02-13T02:24:44.334 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:24:44.335 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:24:44.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.332+0000 7f001d99f6c0 1 -- 10.20.193.27:0/2546226949 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7effec0b5600 msgr2=0x7effec0d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:44.335 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.332+0000 7f001d99f6c0 1 --2- 10.20.193.27:0/2546226949 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7effec0b5600 0x7effec0d5ab0 secure :-1 s=READY pgs=270 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f000c0023e0 tx=0x7f000c005ec0 comp rx=0 tx=0).stop 2026-02-13T02:24:44.336 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.332+0000 7f001d99f6c0 1 -- 10.20.193.27:0/2546226949 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0018115410 msgr2=0x7f001811d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:44.336 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.332+0000 7f001d99f6c0 1 --2- 10.20.193.27:0/2546226949 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0018115410 0x7f001811d840 secure :-1 s=READY pgs=413 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f000000b6c0 tx=0x7f000000bb90 comp rx=0 tx=0).stop 2026-02-13T02:24:44.336 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.332+0000 7f001d99f6c0 1 -- 10.20.193.27:0/2546226949 shutdown_connections 2026-02-13T02:24:44.336 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.332+0000 7f001d99f6c0 1 --2- 10.20.193.27:0/2546226949 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7effec0b5600 0x7effec0d5ab0 unknown :-1 s=CLOSED pgs=270 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:44.336 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.332+0000 7f001d99f6c0 1 --2- 10.20.193.27:0/2546226949 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0018122c00 0x7f00181230a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:44.336 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.332+0000 7f001d99f6c0 1 --2- 10.20.193.27:0/2546226949 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0018115410 0x7f001811d840 unknown :-1 s=CLOSED pgs=413 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:44.336 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.332+0000 7f001d99f6c0 1 --2- 10.20.193.27:0/2546226949 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f001810f200 0x7f001811d300 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:44.336 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.332+0000 7f001d99f6c0 1 -- 10.20.193.27:0/2546226949 >> 10.20.193.27:0/2546226949 conn(0x7f00180ecb00 msgr2=0x7f001810cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:44.336 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.332+0000 7f001d99f6c0 1 -- 10.20.193.27:0/2546226949 shutdown_connections 2026-02-13T02:24:44.336 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:44.332+0000 7f001d99f6c0 1 -- 10.20.193.27:0/2546226949 wait complete. 2026-02-13T02:24:44.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:44 trial028 ceph-mon[26701]: pgmap v261: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:24:44.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:44 trial107 ceph-mon[27422]: pgmap v261: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:24:45.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:45 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2546226949' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:45.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:45 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2546226949' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:45.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:45 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2546226949' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:46.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:46 trial027 ceph-mon[24567]: pgmap v262: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:46.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:46 trial028 ceph-mon[26701]: pgmap v262: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:46.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:46 trial107 ceph-mon[27422]: pgmap v262: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:47.376 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:24:47.496 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:24:47.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.605+0000 7fd7139bf6c0 1 -- 10.20.193.27:0/568834034 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd70c10fbf0 msgr2=0x7fd70c114c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:47.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.605+0000 7fd7139bf6c0 1 --2- 10.20.193.27:0/568834034 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd70c10fbf0 0x7fd70c114c30 secure :-1 s=READY pgs=414 gs=1 cs=0 l=1 c_cookie=2247267882417890045 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd7000099a0 tx=0x7fd70002ed20 comp rx=0 tx=0).stop 2026-02-13T02:24:47.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.605+0000 7fd7139bf6c0 1 -- 10.20.193.27:0/568834034 shutdown_connections 2026-02-13T02:24:47.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.605+0000 7fd7139bf6c0 1 --2- 10.20.193.27:0/568834034 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd70c115410 0x7fd70c175db0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:47.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.605+0000 7fd7139bf6c0 1 --2- 10.20.193.27:0/568834034 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd70c10fbf0 0x7fd70c114c30 unknown :-1 s=CLOSED pgs=414 gs=1 cs=0 l=1 c_cookie=2247267882417890045 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:47.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.605+0000 7fd7139bf6c0 1 --2- 10.20.193.27:0/568834034 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd70c10f200 0x7fd70c10f5d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:47.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.605+0000 7fd7139bf6c0 1 -- 10.20.193.27:0/568834034 >> 10.20.193.27:0/568834034 conn(0x7fd70c0ecb00 msgr2=0x7fd70c10cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:47.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.605+0000 7fd7139bf6c0 1 -- 10.20.193.27:0/568834034 shutdown_connections 2026-02-13T02:24:47.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.605+0000 7fd7139bf6c0 1 -- 10.20.193.27:0/568834034 wait complete. 2026-02-13T02:24:47.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.605+0000 7fd7139bf6c0 1 Processor -- start 2026-02-13T02:24:47.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.605+0000 7fd7139bf6c0 1 -- start start 2026-02-13T02:24:47.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.606+0000 7fd7139bf6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd70c10f200 0x7fd70c0f4c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:47.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.606+0000 7fd7139bf6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd70c115410 0x7fd70c175cb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:47.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.606+0000 7fd7139bf6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd70c0f52c0 0x7fd70c0fa590 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:47.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.606+0000 7fd7139bf6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c0f59b0 con 0x7fd70c115410 2026-02-13T02:24:47.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.606+0000 7fd7139bf6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c0faad0 con 0x7fd70c10f200 2026-02-13T02:24:47.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.606+0000 7fd7139bf6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c0fac90 con 0x7fd70c0f52c0 2026-02-13T02:24:47.612 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.606+0000 7fd710f2f6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd70c115410 0x7fd70c175cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:47.612 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.606+0000 7fd710f2f6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd70c115410 0x7fd70c175cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:56750/0 (socket says 10.20.193.27:56750) 2026-02-13T02:24:47.612 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.606+0000 7fd710f2f6c0 1 -- 10.20.193.27:0/2104571057 learned_addr learned my addr 10.20.193.27:0/2104571057 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:24:47.612 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.606+0000 7fd7117306c0 1 --2- 10.20.193.27:0/2104571057 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd70c10f200 0x7fd70c0f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:47.612 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.606+0000 7fd711f316c0 1 --2- 10.20.193.27:0/2104571057 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd70c0f52c0 0x7fd70c0fa590 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:47.612 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.606+0000 7fd710f2f6c0 1 -- 10.20.193.27:0/2104571057 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd70c0f52c0 msgr2=0x7fd70c0fa590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:47.612 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.606+0000 7fd710f2f6c0 1 --2- 10.20.193.27:0/2104571057 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd70c0f52c0 0x7fd70c0fa590 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:47.612 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.606+0000 7fd710f2f6c0 1 -- 10.20.193.27:0/2104571057 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd70c10f200 msgr2=0x7fd70c0f4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:47.612 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.606+0000 7fd710f2f6c0 1 --2- 10.20.193.27:0/2104571057 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd70c10f200 0x7fd70c0f4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:47.612 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.606+0000 7fd710f2f6c0 1 -- 10.20.193.27:0/2104571057 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd700009590 con 0x7fd70c115410 2026-02-13T02:24:47.612 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.606+0000 7fd710f2f6c0 1 --2- 10.20.193.27:0/2104571057 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd70c115410 0x7fd70c175cb0 secure :-1 s=READY pgs=415 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd700004830 tx=0x7fd70002fbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:47.612 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.606+0000 7fd6fa7fc6c0 1 -- 10.20.193.27:0/2104571057 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd700002ae0 con 0x7fd70c115410 2026-02-13T02:24:47.613 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.606+0000 7fd6fa7fc6c0 1 -- 10.20.193.27:0/2104571057 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd700002c40 con 0x7fd70c115410 2026-02-13T02:24:47.613 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.606+0000 7fd7139bf6c0 1 -- 10.20.193.27:0/2104571057 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd70c0faf10 con 0x7fd70c115410 2026-02-13T02:24:47.613 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.606+0000 7fd6fa7fc6c0 1 -- 10.20.193.27:0/2104571057 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd7000362f0 con 0x7fd70c115410 2026-02-13T02:24:47.613 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.606+0000 7fd7139bf6c0 1 -- 10.20.193.27:0/2104571057 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd70c0fb3b0 con 0x7fd70c115410 2026-02-13T02:24:47.613 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.606+0000 7fd7139bf6c0 1 -- 10.20.193.27:0/2104571057 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd70c1104d0 con 0x7fd70c115410 2026-02-13T02:24:47.614 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.608+0000 7fd6fa7fc6c0 1 -- 10.20.193.27:0/2104571057 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7fd700007810 con 0x7fd70c115410 2026-02-13T02:24:47.614 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.609+0000 7fd6fa7fc6c0 1 --2- 10.20.193.27:0/2104571057 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd6e80b5680 0x7fd6e80d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:47.614 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.609+0000 7fd7117306c0 1 --2- 10.20.193.27:0/2104571057 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd6e80b5680 0x7fd6e80d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:47.614 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.609+0000 7fd6fa7fc6c0 1 -- 10.20.193.27:0/2104571057 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7fd700107c00 con 0x7fd70c115410 2026-02-13T02:24:47.614 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.609+0000 7fd6fa7fc6c0 1 -- 10.20.193.27:0/2104571057 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fd7000cd650 con 0x7fd70c115410 2026-02-13T02:24:47.614 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.609+0000 7fd7117306c0 1 --2- 10.20.193.27:0/2104571057 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd6e80b5680 0x7fd6e80d5b30 secure :-1 s=READY pgs=271 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd6fc006b60 tx=0x7fd6fc006af0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:47.725 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.723+0000 7fd7139bf6c0 1 -- 10.20.193.27:0/2104571057 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd70c10f5d0 con 0x7fd70c115410 2026-02-13T02:24:47.725 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.723+0000 7fd6fa7fc6c0 1 -- 10.20.193.27:0/2104571057 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7fd7000cd830 con 0x7fd70c115410 2026-02-13T02:24:47.725 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:24:47.726 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:24:47.727 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.724+0000 7fd7139bf6c0 1 -- 10.20.193.27:0/2104571057 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd6e80b5680 msgr2=0x7fd6e80d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:47.727 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.724+0000 7fd7139bf6c0 1 --2- 10.20.193.27:0/2104571057 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd6e80b5680 0x7fd6e80d5b30 secure :-1 s=READY pgs=271 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd6fc006b60 tx=0x7fd6fc006af0 comp rx=0 tx=0).stop 2026-02-13T02:24:47.727 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.724+0000 7fd7139bf6c0 1 -- 10.20.193.27:0/2104571057 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd70c115410 msgr2=0x7fd70c175cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:47.727 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.724+0000 7fd7139bf6c0 1 --2- 10.20.193.27:0/2104571057 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd70c115410 0x7fd70c175cb0 secure :-1 s=READY pgs=415 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd700004830 tx=0x7fd70002fbf0 comp rx=0 tx=0).stop 2026-02-13T02:24:47.727 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.724+0000 7fd7139bf6c0 1 -- 10.20.193.27:0/2104571057 shutdown_connections 2026-02-13T02:24:47.727 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.724+0000 7fd7139bf6c0 1 --2- 10.20.193.27:0/2104571057 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd6e80b5680 0x7fd6e80d5b30 unknown :-1 s=CLOSED pgs=271 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:47.727 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.724+0000 7fd7139bf6c0 1 --2- 10.20.193.27:0/2104571057 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd70c0f52c0 0x7fd70c0fa590 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:47.727 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.724+0000 7fd7139bf6c0 1 --2- 10.20.193.27:0/2104571057 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd70c115410 0x7fd70c175cb0 unknown :-1 s=CLOSED pgs=415 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:47.727 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.724+0000 7fd7139bf6c0 1 --2- 10.20.193.27:0/2104571057 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd70c10f200 0x7fd70c0f4c90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:47.727 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.724+0000 7fd7139bf6c0 1 -- 10.20.193.27:0/2104571057 >> 10.20.193.27:0/2104571057 conn(0x7fd70c0ecb00 msgr2=0x7fd70c113d10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:47.728 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.724+0000 7fd7139bf6c0 1 -- 10.20.193.27:0/2104571057 shutdown_connections 2026-02-13T02:24:47.728 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:47.724+0000 7fd7139bf6c0 1 -- 10.20.193.27:0/2104571057 wait complete. 2026-02-13T02:24:48.303 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:48 trial107 ceph-mon[27422]: pgmap v263: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:48.303 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:48 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2104571057' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:48.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:48 trial027 ceph-mon[24567]: pgmap v263: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:48.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:48 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2104571057' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:48.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:48 trial028 ceph-mon[26701]: pgmap v263: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:48.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:48 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2104571057' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:50.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:50 trial027 ceph-mon[24567]: pgmap v264: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:24:50.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:50 trial028 ceph-mon[26701]: pgmap v264: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:24:50.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:50 trial107 ceph-mon[27422]: pgmap v264: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:24:50.769 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:24:50.888 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:24:51.007 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.003+0000 7f4d9dfdc6c0 1 -- 10.20.193.27:0/2212778871 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4d9810fbf0 msgr2=0x7f4d98114c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:51.008 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.003+0000 7f4d9dfdc6c0 1 --2- 10.20.193.27:0/2212778871 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4d9810fbf0 0x7f4d98114c30 secure :-1 s=READY pgs=416 gs=1 cs=0 l=1 c_cookie=11833755378519167156 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4d880099a0 tx=0x7f4d8802ed20 comp rx=0 tx=0).stop 2026-02-13T02:24:51.008 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.003+0000 7f4d9dfdc6c0 1 -- 10.20.193.27:0/2212778871 shutdown_connections 2026-02-13T02:24:51.008 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.003+0000 7f4d9dfdc6c0 1 --2- 10.20.193.27:0/2212778871 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f4d98115410 0x7f4d98175db0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:51.008 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.003+0000 7f4d9dfdc6c0 1 --2- 10.20.193.27:0/2212778871 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4d9810fbf0 0x7f4d98114c30 unknown :-1 s=CLOSED pgs=416 gs=1 cs=0 l=1 c_cookie=11833755378519167156 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:51.008 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.003+0000 7f4d9dfdc6c0 1 --2- 10.20.193.27:0/2212778871 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4d9810f200 0x7f4d9810f5d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:51.008 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.003+0000 7f4d9dfdc6c0 1 -- 10.20.193.27:0/2212778871 >> 10.20.193.27:0/2212778871 conn(0x7f4d980ecb00 msgr2=0x7f4d9810cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:51.008 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.003+0000 7f4d9dfdc6c0 1 -- 10.20.193.27:0/2212778871 shutdown_connections 2026-02-13T02:24:51.008 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.003+0000 7f4d9dfdc6c0 1 -- 10.20.193.27:0/2212778871 wait complete. 2026-02-13T02:24:51.008 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.004+0000 7f4d9dfdc6c0 1 Processor -- start 2026-02-13T02:24:51.008 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.004+0000 7f4d9dfdc6c0 1 -- start start 2026-02-13T02:24:51.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.004+0000 7f4d9dfdc6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f4d9810f200 0x7f4d980f4c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:51.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.004+0000 7f4d9dfdc6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4d98115410 0x7f4d98175cb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:51.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.004+0000 7f4d9dfdc6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4d980f5240 0x7f4d980fa510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:51.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.004+0000 7f4d9dfdc6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4d980f5930 con 0x7f4d98115410 2026-02-13T02:24:51.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.004+0000 7f4d9dfdc6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4d980faa50 con 0x7f4d980f5240 2026-02-13T02:24:51.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.004+0000 7f4d9dfdc6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4d980fac10 con 0x7f4d9810f200 2026-02-13T02:24:51.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.004+0000 7f4d96ffd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4d98115410 0x7f4d98175cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:51.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.004+0000 7f4d96ffd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4d98115410 0x7f4d98175cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:56780/0 (socket says 10.20.193.27:56780) 2026-02-13T02:24:51.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.004+0000 7f4d96ffd6c0 1 -- 10.20.193.27:0/2603316364 learned_addr learned my addr 10.20.193.27:0/2603316364 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:24:51.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.004+0000 7f4d977fe6c0 1 --2- 10.20.193.27:0/2603316364 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f4d9810f200 0x7f4d980f4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:51.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.004+0000 7f4d96ffd6c0 1 -- 10.20.193.27:0/2603316364 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f4d9810f200 msgr2=0x7f4d980f4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:51.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.004+0000 7f4d96ffd6c0 1 --2- 10.20.193.27:0/2603316364 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f4d9810f200 0x7f4d980f4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:51.010 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.004+0000 7f4d96ffd6c0 1 -- 10.20.193.27:0/2603316364 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4d980f5240 msgr2=0x7f4d980fa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:51.010 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.004+0000 7f4d96ffd6c0 1 --2- 10.20.193.27:0/2603316364 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4d980f5240 0x7f4d980fa510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:51.010 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.004+0000 7f4d96ffd6c0 1 -- 10.20.193.27:0/2603316364 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d88009590 con 0x7f4d98115410 2026-02-13T02:24:51.010 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.004+0000 7f4d96ffd6c0 1 --2- 10.20.193.27:0/2603316364 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4d98115410 0x7f4d98175cb0 secure :-1 s=READY pgs=417 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4d8802f230 tx=0x7f4d8802fb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:51.010 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.004+0000 7f4d94ff96c0 1 -- 10.20.193.27:0/2603316364 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4d88002ae0 con 0x7f4d98115410 2026-02-13T02:24:51.010 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.004+0000 7f4d94ff96c0 1 -- 10.20.193.27:0/2603316364 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4d88002c40 con 0x7f4d98115410 2026-02-13T02:24:51.010 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.004+0000 7f4d9dfdc6c0 1 -- 10.20.193.27:0/2603316364 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d980fae90 con 0x7f4d98115410 2026-02-13T02:24:51.010 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.004+0000 7f4d94ff96c0 1 -- 10.20.193.27:0/2603316364 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4d88037070 con 0x7f4d98115410 2026-02-13T02:24:51.010 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.004+0000 7f4d9dfdc6c0 1 -- 10.20.193.27:0/2603316364 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d980fb310 con 0x7f4d98115410 2026-02-13T02:24:51.010 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.005+0000 7f4d9dfdc6c0 1 -- 10.20.193.27:0/2603316364 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d981104d0 con 0x7f4d98115410 2026-02-13T02:24:51.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.006+0000 7f4d94ff96c0 1 -- 10.20.193.27:0/2603316364 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f4d88007710 con 0x7f4d98115410 2026-02-13T02:24:51.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.007+0000 7f4d94ff96c0 1 --2- 10.20.193.27:0/2603316364 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f4d6c0b5600 0x7f4d6c0d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:51.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.007+0000 7f4d94ff96c0 1 -- 10.20.193.27:0/2603316364 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f4d88107b70 con 0x7f4d98115410 2026-02-13T02:24:51.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.007+0000 7f4d977fe6c0 1 --2- 10.20.193.27:0/2603316364 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f4d6c0b5600 0x7f4d6c0d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:51.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.007+0000 7f4d94ff96c0 1 -- 10.20.193.27:0/2603316364 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f4d880cd7e0 con 0x7f4d98115410 2026-02-13T02:24:51.012 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.007+0000 7f4d977fe6c0 1 --2- 10.20.193.27:0/2603316364 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f4d6c0b5600 0x7f4d6c0d5ab0 secure :-1 s=READY pgs=272 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4d84005980 tx=0x7f4d84005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:51.123 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.121+0000 7f4d9dfdc6c0 1 -- 10.20.193.27:0/2603316364 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4d9810f5d0 con 0x7f4d98115410 2026-02-13T02:24:51.124 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.121+0000 7f4d94ff96c0 1 -- 10.20.193.27:0/2603316364 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f4d880cd9c0 con 0x7f4d98115410 2026-02-13T02:24:51.124 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:24:51.124 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:24:51.125 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.122+0000 7f4d9dfdc6c0 1 -- 10.20.193.27:0/2603316364 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f4d6c0b5600 msgr2=0x7f4d6c0d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:51.125 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.122+0000 7f4d9dfdc6c0 1 --2- 10.20.193.27:0/2603316364 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f4d6c0b5600 0x7f4d6c0d5ab0 secure :-1 s=READY pgs=272 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4d84005980 tx=0x7f4d84005910 comp rx=0 tx=0).stop 2026-02-13T02:24:51.125 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.122+0000 7f4d9dfdc6c0 1 -- 10.20.193.27:0/2603316364 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4d98115410 msgr2=0x7f4d98175cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:51.125 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.122+0000 7f4d9dfdc6c0 1 --2- 10.20.193.27:0/2603316364 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4d98115410 0x7f4d98175cb0 secure :-1 s=READY pgs=417 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4d8802f230 tx=0x7f4d8802fb80 comp rx=0 tx=0).stop 2026-02-13T02:24:51.125 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.122+0000 7f4d9dfdc6c0 1 -- 10.20.193.27:0/2603316364 shutdown_connections 2026-02-13T02:24:51.125 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.122+0000 7f4d9dfdc6c0 1 --2- 10.20.193.27:0/2603316364 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f4d6c0b5600 0x7f4d6c0d5ab0 unknown :-1 s=CLOSED pgs=272 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:51.125 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.122+0000 7f4d9dfdc6c0 1 --2- 10.20.193.27:0/2603316364 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4d980f5240 0x7f4d980fa510 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:51.126 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.122+0000 7f4d9dfdc6c0 1 --2- 10.20.193.27:0/2603316364 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4d98115410 0x7f4d98175cb0 unknown :-1 s=CLOSED pgs=417 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:51.126 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.122+0000 7f4d9dfdc6c0 1 --2- 10.20.193.27:0/2603316364 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f4d9810f200 0x7f4d980f4c10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:51.126 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.122+0000 7f4d9dfdc6c0 1 -- 10.20.193.27:0/2603316364 >> 10.20.193.27:0/2603316364 conn(0x7f4d980ecb00 msgr2=0x7f4d9810d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:51.126 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.122+0000 7f4d9dfdc6c0 1 -- 10.20.193.27:0/2603316364 shutdown_connections 2026-02-13T02:24:51.126 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:51.122+0000 7f4d9dfdc6c0 1 -- 10.20.193.27:0/2603316364 wait complete. 2026-02-13T02:24:52.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:52 trial027 ceph-mon[24567]: pgmap v265: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:52.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:52 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2603316364' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:52.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:52 trial028 ceph-mon[26701]: pgmap v265: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:52.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:52 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2603316364' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:52.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:52 trial107 ceph-mon[27422]: pgmap v265: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:52.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:52 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2603316364' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:54.167 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:24:54.284 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:24:54.306 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:54 trial027 ceph-mon[24567]: pgmap v266: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:24:54.346 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:54 trial107 ceph-mon[27422]: pgmap v266: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:24:54.409 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.404+0000 7ffb7433b6c0 1 -- 10.20.193.27:0/2326049078 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ffb6c115410 msgr2=0x7ffb6c175db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:54.409 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.404+0000 7ffb7433b6c0 1 --2- 10.20.193.27:0/2326049078 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ffb6c115410 0x7ffb6c175db0 secure :-1 s=READY pgs=418 gs=1 cs=0 l=1 c_cookie=14025496372520658829 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffb6800aa70 tx=0x7ffb6802fde0 comp rx=0 tx=0).stop 2026-02-13T02:24:54.409 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.405+0000 7ffb7433b6c0 1 -- 10.20.193.27:0/2326049078 shutdown_connections 2026-02-13T02:24:54.409 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.405+0000 7ffb7433b6c0 1 --2- 10.20.193.27:0/2326049078 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ffb6c115410 0x7ffb6c175db0 unknown :-1 s=CLOSED pgs=418 gs=1 cs=0 l=1 c_cookie=14025496372520658829 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:54.410 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.405+0000 7ffb7433b6c0 1 --2- 10.20.193.27:0/2326049078 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffb6c10fbf0 0x7ffb6c114c30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:54.410 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.405+0000 7ffb7433b6c0 1 --2- 10.20.193.27:0/2326049078 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ffb6c10f200 0x7ffb6c10f5d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:54.410 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.405+0000 7ffb7433b6c0 1 -- 10.20.193.27:0/2326049078 >> 10.20.193.27:0/2326049078 conn(0x7ffb6c0ecb00 msgr2=0x7ffb6c10cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:54.410 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.405+0000 7ffb7433b6c0 1 -- 10.20.193.27:0/2326049078 shutdown_connections 2026-02-13T02:24:54.410 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.405+0000 7ffb7433b6c0 1 -- 10.20.193.27:0/2326049078 wait complete. 2026-02-13T02:24:54.410 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.405+0000 7ffb7433b6c0 1 Processor -- start 2026-02-13T02:24:54.410 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.405+0000 7ffb7433b6c0 1 -- start start 2026-02-13T02:24:54.410 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.405+0000 7ffb7433b6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ffb6c10f200 0x7ffb6c11d290 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:54.410 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.405+0000 7ffb7433b6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffb6c115410 0x7ffb6c11d7d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:54.410 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.405+0000 7ffb7433b6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ffb6c122b90 0x7ffb6c123030 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:54.410 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.405+0000 7ffb7433b6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6c1235a0 con 0x7ffb6c122b90 2026-02-13T02:24:54.410 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.405+0000 7ffb7433b6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6c123710 con 0x7ffb6c115410 2026-02-13T02:24:54.411 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.405+0000 7ffb7433b6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6c1238d0 con 0x7ffb6c10f200 2026-02-13T02:24:54.411 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.405+0000 7ffb728ad6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ffb6c122b90 0x7ffb6c123030 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:54.411 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.405+0000 7ffb728ad6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ffb6c122b90 0x7ffb6c123030 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:56784/0 (socket says 10.20.193.27:56784) 2026-02-13T02:24:54.411 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.405+0000 7ffb728ad6c0 1 -- 10.20.193.27:0/3103490678 learned_addr learned my addr 10.20.193.27:0/3103490678 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:24:54.411 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.405+0000 7ffb720ac6c0 1 --2- 10.20.193.27:0/3103490678 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ffb6c10f200 0x7ffb6c11d290 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:54.411 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.405+0000 7ffb718ab6c0 1 --2- 10.20.193.27:0/3103490678 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffb6c115410 0x7ffb6c11d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:54.411 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.405+0000 7ffb728ad6c0 1 -- 10.20.193.27:0/3103490678 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ffb6c10f200 msgr2=0x7ffb6c11d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:54.411 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.405+0000 7ffb728ad6c0 1 --2- 10.20.193.27:0/3103490678 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ffb6c10f200 0x7ffb6c11d290 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:54.411 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.405+0000 7ffb728ad6c0 1 -- 10.20.193.27:0/3103490678 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffb6c115410 msgr2=0x7ffb6c11d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:54.411 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.405+0000 7ffb728ad6c0 1 --2- 10.20.193.27:0/3103490678 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffb6c115410 0x7ffb6c11d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:54.411 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.405+0000 7ffb728ad6c0 1 -- 10.20.193.27:0/3103490678 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb6800a660 con 0x7ffb6c122b90 2026-02-13T02:24:54.411 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.405+0000 7ffb728ad6c0 1 --2- 10.20.193.27:0/3103490678 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ffb6c122b90 0x7ffb6c123030 secure :-1 s=READY pgs=419 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffb680054b0 tx=0x7ffb680054e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:54.412 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.406+0000 7ffb537fe6c0 1 -- 10.20.193.27:0/3103490678 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ffb68003a30 con 0x7ffb6c122b90 2026-02-13T02:24:54.412 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.406+0000 7ffb537fe6c0 1 -- 10.20.193.27:0/3103490678 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ffb68003b90 con 0x7ffb6c122b90 2026-02-13T02:24:54.412 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.406+0000 7ffb7433b6c0 1 -- 10.20.193.27:0/3103490678 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb6c163d20 con 0x7ffb6c122b90 2026-02-13T02:24:54.412 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.406+0000 7ffb537fe6c0 1 -- 10.20.193.27:0/3103490678 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ffb680382f0 con 0x7ffb6c122b90 2026-02-13T02:24:54.412 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.406+0000 7ffb7433b6c0 1 -- 10.20.193.27:0/3103490678 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb6c164150 con 0x7ffb6c122b90 2026-02-13T02:24:54.412 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.406+0000 7ffb7433b6c0 1 -- 10.20.193.27:0/3103490678 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb6c1104d0 con 0x7ffb6c122b90 2026-02-13T02:24:54.413 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.407+0000 7ffb537fe6c0 1 -- 10.20.193.27:0/3103490678 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7ffb68008260 con 0x7ffb6c122b90 2026-02-13T02:24:54.413 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.408+0000 7ffb537fe6c0 1 --2- 10.20.193.27:0/3103490678 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ffb480b5680 0x7ffb480d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:54.413 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.408+0000 7ffb720ac6c0 1 --2- 10.20.193.27:0/3103490678 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ffb480b5680 0x7ffb480d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:54.413 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.408+0000 7ffb537fe6c0 1 -- 10.20.193.27:0/3103490678 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7ffb68108e60 con 0x7ffb6c122b90 2026-02-13T02:24:54.413 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.409+0000 7ffb537fe6c0 1 -- 10.20.193.27:0/3103490678 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7ffb680cea90 con 0x7ffb6c122b90 2026-02-13T02:24:54.414 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.409+0000 7ffb720ac6c0 1 --2- 10.20.193.27:0/3103490678 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ffb480b5680 0x7ffb480d5b30 secure :-1 s=READY pgs=273 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffb5c005980 tx=0x7ffb5c005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:54.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:54 trial028 ceph-mon[26701]: pgmap v266: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:24:54.525 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.522+0000 7ffb7433b6c0 1 -- 10.20.193.27:0/3103490678 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ffb6c11e3e0 con 0x7ffb6c122b90 2026-02-13T02:24:54.525 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.522+0000 7ffb537fe6c0 1 -- 10.20.193.27:0/3103490678 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7ffb680cec70 con 0x7ffb6c122b90 2026-02-13T02:24:54.525 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:24:54.525 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:24:54.526 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.523+0000 7ffb7433b6c0 1 -- 10.20.193.27:0/3103490678 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ffb480b5680 msgr2=0x7ffb480d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:54.526 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.523+0000 7ffb7433b6c0 1 --2- 10.20.193.27:0/3103490678 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ffb480b5680 0x7ffb480d5b30 secure :-1 s=READY pgs=273 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffb5c005980 tx=0x7ffb5c005910 comp rx=0 tx=0).stop 2026-02-13T02:24:54.526 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.523+0000 7ffb7433b6c0 1 -- 10.20.193.27:0/3103490678 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ffb6c122b90 msgr2=0x7ffb6c123030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:54.526 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.523+0000 7ffb7433b6c0 1 --2- 10.20.193.27:0/3103490678 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ffb6c122b90 0x7ffb6c123030 secure :-1 s=READY pgs=419 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffb680054b0 tx=0x7ffb680054e0 comp rx=0 tx=0).stop 2026-02-13T02:24:54.527 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.523+0000 7ffb7433b6c0 1 -- 10.20.193.27:0/3103490678 shutdown_connections 2026-02-13T02:24:54.527 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.523+0000 7ffb7433b6c0 1 --2- 10.20.193.27:0/3103490678 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ffb480b5680 0x7ffb480d5b30 unknown :-1 s=CLOSED pgs=273 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:54.527 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.523+0000 7ffb7433b6c0 1 --2- 10.20.193.27:0/3103490678 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ffb6c122b90 0x7ffb6c123030 unknown :-1 s=CLOSED pgs=419 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:54.527 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.523+0000 7ffb7433b6c0 1 --2- 10.20.193.27:0/3103490678 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffb6c115410 0x7ffb6c11d7d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:54.527 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.523+0000 7ffb7433b6c0 1 --2- 10.20.193.27:0/3103490678 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ffb6c10f200 0x7ffb6c11d290 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:54.527 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.523+0000 7ffb7433b6c0 1 -- 10.20.193.27:0/3103490678 >> 10.20.193.27:0/3103490678 conn(0x7ffb6c0ecb00 msgr2=0x7ffb6c10cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:54.527 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.523+0000 7ffb7433b6c0 1 -- 10.20.193.27:0/3103490678 shutdown_connections 2026-02-13T02:24:54.527 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:54.523+0000 7ffb7433b6c0 1 -- 10.20.193.27:0/3103490678 wait complete. 2026-02-13T02:24:55.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:55 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3103490678' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:55.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:55 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3103490678' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:55.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:55 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3103490678' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:56.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:56 trial027 ceph-mon[24567]: pgmap v267: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:56.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:56 trial028 ceph-mon[26701]: pgmap v267: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:56.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:56 trial107 ceph-mon[27422]: pgmap v267: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:57.567 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:24:57.687 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:24:57.800 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.796+0000 7fd9799596c0 1 -- 10.20.193.27:0/698370178 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd97410fbf0 msgr2=0x7fd974114c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:57.801 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.796+0000 7fd9799596c0 1 --2- 10.20.193.27:0/698370178 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd97410fbf0 0x7fd974114c30 secure :-1 s=READY pgs=420 gs=1 cs=0 l=1 c_cookie=9205940769234332313 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd95c0099a0 tx=0x7fd95c02ed50 comp rx=0 tx=0).stop 2026-02-13T02:24:57.801 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.796+0000 7fd9799596c0 1 -- 10.20.193.27:0/698370178 shutdown_connections 2026-02-13T02:24:57.801 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.796+0000 7fd9799596c0 1 --2- 10.20.193.27:0/698370178 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd974115410 0x7fd974175db0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:57.801 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.796+0000 7fd9799596c0 1 --2- 10.20.193.27:0/698370178 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd97410fbf0 0x7fd974114c30 unknown :-1 s=CLOSED pgs=420 gs=1 cs=0 l=1 c_cookie=9205940769234332313 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:57.801 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.796+0000 7fd9799596c0 1 --2- 10.20.193.27:0/698370178 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd97410f200 0x7fd97410f5d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:57.801 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.796+0000 7fd9799596c0 1 -- 10.20.193.27:0/698370178 >> 10.20.193.27:0/698370178 conn(0x7fd9740ecb00 msgr2=0x7fd97410cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:57.801 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.796+0000 7fd9799596c0 1 -- 10.20.193.27:0/698370178 shutdown_connections 2026-02-13T02:24:57.801 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.796+0000 7fd9799596c0 1 -- 10.20.193.27:0/698370178 wait complete. 2026-02-13T02:24:57.801 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.797+0000 7fd9799596c0 1 Processor -- start 2026-02-13T02:24:57.801 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.797+0000 7fd9799596c0 1 -- start start 2026-02-13T02:24:57.801 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.797+0000 7fd9799596c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd97410f200 0x7fd9740f4c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:57.802 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.797+0000 7fd9799596c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd974115410 0x7fd974175cb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:57.802 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.797+0000 7fd9799596c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd9740f5240 0x7fd9740fa510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:57.802 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.797+0000 7fd9799596c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd9740f58a0 con 0x7fd974115410 2026-02-13T02:24:57.802 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.797+0000 7fd9799596c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd9740faa50 con 0x7fd97410f200 2026-02-13T02:24:57.802 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.797+0000 7fd9799596c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd9740fac10 con 0x7fd9740f5240 2026-02-13T02:24:57.802 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.797+0000 7fd9727fc6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd974115410 0x7fd974175cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:57.802 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.797+0000 7fd9727fc6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd974115410 0x7fd974175cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:52572/0 (socket says 10.20.193.27:52572) 2026-02-13T02:24:57.802 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.797+0000 7fd9727fc6c0 1 -- 10.20.193.27:0/1820812876 learned_addr learned my addr 10.20.193.27:0/1820812876 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:24:57.802 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.797+0000 7fd9737fe6c0 1 --2- 10.20.193.27:0/1820812876 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd9740f5240 0x7fd9740fa510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:57.802 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.797+0000 7fd9727fc6c0 1 -- 10.20.193.27:0/1820812876 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd9740f5240 msgr2=0x7fd9740fa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:57.802 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.797+0000 7fd9727fc6c0 1 --2- 10.20.193.27:0/1820812876 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd9740f5240 0x7fd9740fa510 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:57.803 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.797+0000 7fd9727fc6c0 1 -- 10.20.193.27:0/1820812876 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd97410f200 msgr2=0x7fd9740f4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:57.803 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.797+0000 7fd9727fc6c0 1 --2- 10.20.193.27:0/1820812876 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd97410f200 0x7fd9740f4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:57.803 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.797+0000 7fd9727fc6c0 1 -- 10.20.193.27:0/1820812876 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd95c009590 con 0x7fd974115410 2026-02-13T02:24:57.803 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.797+0000 7fd9737fe6c0 1 --2- 10.20.193.27:0/1820812876 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd9740f5240 0x7fd9740fa510 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:24:57.803 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.797+0000 7fd9727fc6c0 1 --2- 10.20.193.27:0/1820812876 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd974115410 0x7fd974175cb0 secure :-1 s=READY pgs=421 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd95c0023a0 tx=0x7fd95c02fb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:57.803 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.797+0000 7fd9789536c0 1 -- 10.20.193.27:0/1820812876 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd95c002ae0 con 0x7fd974115410 2026-02-13T02:24:57.803 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.797+0000 7fd9789536c0 1 -- 10.20.193.27:0/1820812876 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd95c002c40 con 0x7fd974115410 2026-02-13T02:24:57.803 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.797+0000 7fd9799596c0 1 -- 10.20.193.27:0/1820812876 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9740fae30 con 0x7fd974115410 2026-02-13T02:24:57.803 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.797+0000 7fd9789536c0 1 -- 10.20.193.27:0/1820812876 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd95c037070 con 0x7fd974115410 2026-02-13T02:24:57.803 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.797+0000 7fd9799596c0 1 -- 10.20.193.27:0/1820812876 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9740fb300 con 0x7fd974115410 2026-02-13T02:24:57.803 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.798+0000 7fd9799596c0 1 -- 10.20.193.27:0/1820812876 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9741104d0 con 0x7fd974115410 2026-02-13T02:24:57.805 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.799+0000 7fd9789536c0 1 -- 10.20.193.27:0/1820812876 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7fd95c0071c0 con 0x7fd974115410 2026-02-13T02:24:57.805 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.800+0000 7fd9789536c0 1 --2- 10.20.193.27:0/1820812876 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd9480b5680 0x7fd9480d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:24:57.805 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.800+0000 7fd972ffd6c0 1 --2- 10.20.193.27:0/1820812876 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd9480b5680 0x7fd9480d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:24:57.805 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.800+0000 7fd9789536c0 1 -- 10.20.193.27:0/1820812876 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7fd95c107c90 con 0x7fd974115410 2026-02-13T02:24:57.805 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.800+0000 7fd972ffd6c0 1 --2- 10.20.193.27:0/1820812876 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd9480b5680 0x7fd9480d5b30 secure :-1 s=READY pgs=274 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd968004750 tx=0x7fd968005d30 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:24:57.805 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.800+0000 7fd9789536c0 1 -- 10.20.193.27:0/1820812876 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fd95c0ce1b0 con 0x7fd974115410 2026-02-13T02:24:57.917 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.914+0000 7fd9799596c0 1 -- 10.20.193.27:0/1820812876 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd97410f5d0 con 0x7fd974115410 2026-02-13T02:24:57.917 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.914+0000 7fd9789536c0 1 -- 10.20.193.27:0/1820812876 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7fd95c094810 con 0x7fd974115410 2026-02-13T02:24:57.917 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:24:57.917 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:24:57.918 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.915+0000 7fd9799596c0 1 -- 10.20.193.27:0/1820812876 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd9480b5680 msgr2=0x7fd9480d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:57.918 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.915+0000 7fd9799596c0 1 --2- 10.20.193.27:0/1820812876 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd9480b5680 0x7fd9480d5b30 secure :-1 s=READY pgs=274 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd968004750 tx=0x7fd968005d30 comp rx=0 tx=0).stop 2026-02-13T02:24:57.918 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.915+0000 7fd9799596c0 1 -- 10.20.193.27:0/1820812876 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd974115410 msgr2=0x7fd974175cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:24:57.918 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.915+0000 7fd9799596c0 1 --2- 10.20.193.27:0/1820812876 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd974115410 0x7fd974175cb0 secure :-1 s=READY pgs=421 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd95c0023a0 tx=0x7fd95c02fb70 comp rx=0 tx=0).stop 2026-02-13T02:24:57.918 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.915+0000 7fd9799596c0 1 -- 10.20.193.27:0/1820812876 shutdown_connections 2026-02-13T02:24:57.919 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.915+0000 7fd9799596c0 1 --2- 10.20.193.27:0/1820812876 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd9480b5680 0x7fd9480d5b30 unknown :-1 s=CLOSED pgs=274 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:57.919 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.915+0000 7fd9799596c0 1 --2- 10.20.193.27:0/1820812876 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd9740f5240 0x7fd9740fa510 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:57.919 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.915+0000 7fd9799596c0 1 --2- 10.20.193.27:0/1820812876 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd974115410 0x7fd974175cb0 unknown :-1 s=CLOSED pgs=421 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:57.919 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.915+0000 7fd9799596c0 1 --2- 10.20.193.27:0/1820812876 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd97410f200 0x7fd9740f4c10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:24:57.919 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.915+0000 7fd9799596c0 1 -- 10.20.193.27:0/1820812876 >> 10.20.193.27:0/1820812876 conn(0x7fd9740ecb00 msgr2=0x7fd97410d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:24:57.919 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.915+0000 7fd9799596c0 1 -- 10.20.193.27:0/1820812876 shutdown_connections 2026-02-13T02:24:57.919 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:24:57.915+0000 7fd9799596c0 1 -- 10.20.193.27:0/1820812876 wait complete. 2026-02-13T02:24:58.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:58 trial027 ceph-mon[24567]: pgmap v268: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:58.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:24:58 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1820812876' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:58.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:58 trial028 ceph-mon[26701]: pgmap v268: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:58.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:24:58 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1820812876' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:24:58.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:58 trial107 ceph-mon[27422]: pgmap v268: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:24:58.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:24:58 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1820812876' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:00.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:00 trial027 ceph-mon[24567]: pgmap v269: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:00.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:00 trial028 ceph-mon[26701]: pgmap v269: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:00.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:00 trial107 ceph-mon[27422]: pgmap v269: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:00.958 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:25:01.078 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:25:01.190 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.186+0000 7fda75b946c0 1 -- 10.20.193.27:0/3437786132 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fda7010f200 msgr2=0x7fda7010f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:01.191 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.186+0000 7fda75b946c0 1 --2- 10.20.193.27:0/3437786132 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fda7010f200 0x7fda7010f5d0 secure :-1 s=READY pgs=422 gs=1 cs=0 l=1 c_cookie=11909635034975081338 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fda5c0099a0 tx=0x7fda5c02ed20 comp rx=0 tx=0).stop 2026-02-13T02:25:01.191 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.186+0000 7fda75b946c0 1 -- 10.20.193.27:0/3437786132 shutdown_connections 2026-02-13T02:25:01.191 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.186+0000 7fda75b946c0 1 --2- 10.20.193.27:0/3437786132 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fda70115410 0x7fda70175db0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:01.191 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.186+0000 7fda75b946c0 1 --2- 10.20.193.27:0/3437786132 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fda7010fbf0 0x7fda70114c30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:01.191 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.186+0000 7fda75b946c0 1 --2- 10.20.193.27:0/3437786132 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fda7010f200 0x7fda7010f5d0 unknown :-1 s=CLOSED pgs=422 gs=1 cs=0 l=1 c_cookie=11909635034975081338 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:01.191 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.186+0000 7fda75b946c0 1 -- 10.20.193.27:0/3437786132 >> 10.20.193.27:0/3437786132 conn(0x7fda700ecb00 msgr2=0x7fda7010cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:01.191 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.186+0000 7fda75b946c0 1 -- 10.20.193.27:0/3437786132 shutdown_connections 2026-02-13T02:25:01.191 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.186+0000 7fda75b946c0 1 -- 10.20.193.27:0/3437786132 wait complete. 2026-02-13T02:25:01.191 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.186+0000 7fda75b946c0 1 Processor -- start 2026-02-13T02:25:01.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.186+0000 7fda75b946c0 1 -- start start 2026-02-13T02:25:01.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.186+0000 7fda75b946c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fda7010f200 0x7fda7011d260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:01.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.186+0000 7fda75b946c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fda70115410 0x7fda7011d7a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:01.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.186+0000 7fda75b946c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fda70122b60 0x7fda70123000 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:01.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.186+0000 7fda75b946c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fda70123570 con 0x7fda7010f200 2026-02-13T02:25:01.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.186+0000 7fda75b946c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fda701236e0 con 0x7fda70122b60 2026-02-13T02:25:01.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.186+0000 7fda75b946c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fda701238a0 con 0x7fda70115410 2026-02-13T02:25:01.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.186+0000 7fda6f7fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fda7010f200 0x7fda7011d260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:01.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.187+0000 7fda6f7fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fda7010f200 0x7fda7011d260 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:52596/0 (socket says 10.20.193.27:52596) 2026-02-13T02:25:01.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.187+0000 7fda6f7fe6c0 1 -- 10.20.193.27:0/2902160537 learned_addr learned my addr 10.20.193.27:0/2902160537 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:25:01.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.187+0000 7fda6ffff6c0 1 --2- 10.20.193.27:0/2902160537 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fda70122b60 0x7fda70123000 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:01.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.187+0000 7fda6effd6c0 1 --2- 10.20.193.27:0/2902160537 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fda70115410 0x7fda7011d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:01.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.187+0000 7fda6f7fe6c0 1 -- 10.20.193.27:0/2902160537 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fda70115410 msgr2=0x7fda7011d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:01.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.187+0000 7fda6f7fe6c0 1 --2- 10.20.193.27:0/2902160537 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fda70115410 0x7fda7011d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:01.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.187+0000 7fda6f7fe6c0 1 -- 10.20.193.27:0/2902160537 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fda70122b60 msgr2=0x7fda70123000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:01.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.187+0000 7fda6f7fe6c0 1 --2- 10.20.193.27:0/2902160537 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fda70122b60 0x7fda70123000 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:01.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.187+0000 7fda6f7fe6c0 1 -- 10.20.193.27:0/2902160537 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda5c009590 con 0x7fda7010f200 2026-02-13T02:25:01.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.187+0000 7fda6effd6c0 1 --2- 10.20.193.27:0/2902160537 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fda70115410 0x7fda7011d7a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:25:01.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.187+0000 7fda6f7fe6c0 1 --2- 10.20.193.27:0/2902160537 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fda7010f200 0x7fda7011d260 secure :-1 s=READY pgs=423 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fda5c02f230 tx=0x7fda5c02fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:01.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.187+0000 7fda6cff96c0 1 -- 10.20.193.27:0/2902160537 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fda5c002ae0 con 0x7fda7010f200 2026-02-13T02:25:01.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.187+0000 7fda75b946c0 1 -- 10.20.193.27:0/2902160537 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda70163cf0 con 0x7fda7010f200 2026-02-13T02:25:01.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.187+0000 7fda6cff96c0 1 -- 10.20.193.27:0/2902160537 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fda5c007680 con 0x7fda7010f200 2026-02-13T02:25:01.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.187+0000 7fda6cff96c0 1 -- 10.20.193.27:0/2902160537 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fda5c037520 con 0x7fda7010f200 2026-02-13T02:25:01.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.187+0000 7fda75b946c0 1 -- 10.20.193.27:0/2902160537 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda70164120 con 0x7fda7010f200 2026-02-13T02:25:01.194 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.187+0000 7fda75b946c0 1 -- 10.20.193.27:0/2902160537 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda701104d0 con 0x7fda7010f200 2026-02-13T02:25:01.194 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.189+0000 7fda6cff96c0 1 -- 10.20.193.27:0/2902160537 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7fda5c007cb0 con 0x7fda7010f200 2026-02-13T02:25:01.195 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.189+0000 7fda6cff96c0 1 --2- 10.20.193.27:0/2902160537 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fda440b5600 0x7fda440d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:01.195 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.189+0000 7fda6cff96c0 1 -- 10.20.193.27:0/2902160537 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7fda5c107d10 con 0x7fda7010f200 2026-02-13T02:25:01.195 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.189+0000 7fda6effd6c0 1 --2- 10.20.193.27:0/2902160537 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fda440b5600 0x7fda440d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:01.195 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.189+0000 7fda6effd6c0 1 --2- 10.20.193.27:0/2902160537 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fda440b5600 0x7fda440d5ab0 secure :-1 s=READY pgs=275 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fda70123bf0 tx=0x7fda60009260 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:01.195 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.190+0000 7fda6cff96c0 1 -- 10.20.193.27:0/2902160537 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fda5c0ca7d0 con 0x7fda7010f200 2026-02-13T02:25:01.310 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.308+0000 7fda75b946c0 1 -- 10.20.193.27:0/2902160537 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fda7010f5d0 con 0x7fda7010f200 2026-02-13T02:25:01.311 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.308+0000 7fda6cff96c0 1 -- 10.20.193.27:0/2902160537 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7fda5c0cd980 con 0x7fda7010f200 2026-02-13T02:25:01.311 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:25:01.311 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:25:01.312 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.309+0000 7fda75b946c0 1 -- 10.20.193.27:0/2902160537 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fda440b5600 msgr2=0x7fda440d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:01.312 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.309+0000 7fda75b946c0 1 --2- 10.20.193.27:0/2902160537 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fda440b5600 0x7fda440d5ab0 secure :-1 s=READY pgs=275 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fda70123bf0 tx=0x7fda60009260 comp rx=0 tx=0).stop 2026-02-13T02:25:01.312 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.309+0000 7fda75b946c0 1 -- 10.20.193.27:0/2902160537 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fda7010f200 msgr2=0x7fda7011d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:01.312 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.309+0000 7fda75b946c0 1 --2- 10.20.193.27:0/2902160537 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fda7010f200 0x7fda7011d260 secure :-1 s=READY pgs=423 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fda5c02f230 tx=0x7fda5c02fc00 comp rx=0 tx=0).stop 2026-02-13T02:25:01.313 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.309+0000 7fda75b946c0 1 -- 10.20.193.27:0/2902160537 shutdown_connections 2026-02-13T02:25:01.313 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.309+0000 7fda75b946c0 1 --2- 10.20.193.27:0/2902160537 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fda440b5600 0x7fda440d5ab0 unknown :-1 s=CLOSED pgs=275 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:01.313 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.309+0000 7fda75b946c0 1 --2- 10.20.193.27:0/2902160537 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fda70122b60 0x7fda70123000 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:01.313 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.309+0000 7fda75b946c0 1 --2- 10.20.193.27:0/2902160537 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fda70115410 0x7fda7011d7a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:01.313 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.309+0000 7fda75b946c0 1 --2- 10.20.193.27:0/2902160537 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fda7010f200 0x7fda7011d260 unknown :-1 s=CLOSED pgs=423 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:01.313 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.309+0000 7fda75b946c0 1 -- 10.20.193.27:0/2902160537 >> 10.20.193.27:0/2902160537 conn(0x7fda700ecb00 msgr2=0x7fda7010cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:01.313 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.309+0000 7fda75b946c0 1 -- 10.20.193.27:0/2902160537 shutdown_connections 2026-02-13T02:25:01.313 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:01.309+0000 7fda75b946c0 1 -- 10.20.193.27:0/2902160537 wait complete. 2026-02-13T02:25:02.295 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:02 trial028 ceph-mon[26701]: pgmap v270: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:02.295 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:02 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2902160537' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:02.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:02 trial027 ceph-mon[24567]: pgmap v270: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:02.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:02 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2902160537' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:02.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:02 trial107 ceph-mon[27422]: pgmap v270: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:02.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:02 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2902160537' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:04.368 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:25:04.393 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:04 trial027 ceph-mon[24567]: pgmap v271: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:04.487 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:25:04.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:04 trial028 ceph-mon[26701]: pgmap v271: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:04.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:04 trial107 ceph-mon[27422]: pgmap v271: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:04.607 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.604+0000 7f1461a646c0 1 -- 10.20.193.27:0/2399760567 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f145c10f200 msgr2=0x7f145c10f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:04.607 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.604+0000 7f1461a646c0 1 --2- 10.20.193.27:0/2399760567 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f145c10f200 0x7f145c10f5d0 secure :-1 s=READY pgs=424 gs=1 cs=0 l=1 c_cookie=58746345933238596 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f14500099a0 tx=0x7f145002ed20 comp rx=0 tx=0).stop 2026-02-13T02:25:04.607 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.604+0000 7f1461a646c0 1 -- 10.20.193.27:0/2399760567 shutdown_connections 2026-02-13T02:25:04.607 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.604+0000 7f1461a646c0 1 --2- 10.20.193.27:0/2399760567 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f145c115410 0x7f145c175db0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:04.608 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.604+0000 7f1461a646c0 1 --2- 10.20.193.27:0/2399760567 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f145c10fbf0 0x7f145c114c30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:04.608 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.604+0000 7f1461a646c0 1 --2- 10.20.193.27:0/2399760567 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f145c10f200 0x7f145c10f5d0 unknown :-1 s=CLOSED pgs=424 gs=1 cs=0 l=1 c_cookie=58746345933238596 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:04.608 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.604+0000 7f1461a646c0 1 -- 10.20.193.27:0/2399760567 >> 10.20.193.27:0/2399760567 conn(0x7f145c0ecb00 msgr2=0x7f145c10cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:04.608 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.604+0000 7f1461a646c0 1 -- 10.20.193.27:0/2399760567 shutdown_connections 2026-02-13T02:25:04.608 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.604+0000 7f1461a646c0 1 -- 10.20.193.27:0/2399760567 wait complete. 2026-02-13T02:25:04.608 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.604+0000 7f1461a646c0 1 Processor -- start 2026-02-13T02:25:04.608 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.604+0000 7f1461a646c0 1 -- start start 2026-02-13T02:25:04.608 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.604+0000 7f1461a646c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f145c10f200 0x7f145c11d2f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:04.608 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.604+0000 7f1461a646c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f145c115410 0x7f145c11d830 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:04.608 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.604+0000 7f1461a646c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f145c122bf0 0x7f145c123090 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:04.608 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.604+0000 7f1461a646c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f145c123600 con 0x7f145c115410 2026-02-13T02:25:04.609 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.604+0000 7f1461a646c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f145c123770 con 0x7f145c10f200 2026-02-13T02:25:04.609 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.604+0000 7f1461a646c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f145c123930 con 0x7f145c122bf0 2026-02-13T02:25:04.609 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.604+0000 7f145a7fc6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f145c115410 0x7f145c11d830 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:04.609 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.604+0000 7f145a7fc6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f145c115410 0x7f145c11d830 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:52608/0 (socket says 10.20.193.27:52608) 2026-02-13T02:25:04.609 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.604+0000 7f145a7fc6c0 1 -- 10.20.193.27:0/1853101020 learned_addr learned my addr 10.20.193.27:0/1853101020 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:25:04.609 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.604+0000 7f145b7fe6c0 1 --2- 10.20.193.27:0/1853101020 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f145c122bf0 0x7f145c123090 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:04.609 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.604+0000 7f145affd6c0 1 --2- 10.20.193.27:0/1853101020 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f145c10f200 0x7f145c11d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:04.609 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.604+0000 7f145a7fc6c0 1 -- 10.20.193.27:0/1853101020 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f145c122bf0 msgr2=0x7f145c123090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:04.609 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.604+0000 7f145a7fc6c0 1 --2- 10.20.193.27:0/1853101020 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f145c122bf0 0x7f145c123090 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:04.609 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.604+0000 7f145a7fc6c0 1 -- 10.20.193.27:0/1853101020 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f145c10f200 msgr2=0x7f145c11d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:04.609 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.604+0000 7f145a7fc6c0 1 --2- 10.20.193.27:0/1853101020 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f145c10f200 0x7f145c11d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:04.609 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.604+0000 7f145a7fc6c0 1 -- 10.20.193.27:0/1853101020 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1450009590 con 0x7f145c115410 2026-02-13T02:25:04.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.604+0000 7f145affd6c0 1 --2- 10.20.193.27:0/1853101020 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f145c10f200 0x7f145c11d2f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:25:04.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.605+0000 7f145a7fc6c0 1 --2- 10.20.193.27:0/1853101020 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f145c115410 0x7f145c11d830 secure :-1 s=READY pgs=425 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f144400b6c0 tx=0x7f144400bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:04.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.605+0000 7f1460a5e6c0 1 -- 10.20.193.27:0/1853101020 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f14440044a0 con 0x7f145c115410 2026-02-13T02:25:04.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.605+0000 7f1460a5e6c0 1 -- 10.20.193.27:0/1853101020 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1444009030 con 0x7f145c115410 2026-02-13T02:25:04.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.605+0000 7f1460a5e6c0 1 -- 10.20.193.27:0/1853101020 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f144400cc10 con 0x7f145c115410 2026-02-13T02:25:04.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.605+0000 7f1461a646c0 1 -- 10.20.193.27:0/1853101020 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f145c163de0 con 0x7f145c115410 2026-02-13T02:25:04.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.605+0000 7f1461a646c0 1 -- 10.20.193.27:0/1853101020 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f145c164270 con 0x7f145c115410 2026-02-13T02:25:04.610 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.605+0000 7f1461a646c0 1 -- 10.20.193.27:0/1853101020 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f145c1104d0 con 0x7f145c115410 2026-02-13T02:25:04.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.606+0000 7f1460a5e6c0 1 -- 10.20.193.27:0/1853101020 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f14440026b0 con 0x7f145c115410 2026-02-13T02:25:04.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.607+0000 7f1460a5e6c0 1 --2- 10.20.193.27:0/1853101020 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f14300b5680 0x7f14300d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:04.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.607+0000 7f145affd6c0 1 --2- 10.20.193.27:0/1853101020 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f14300b5680 0x7f14300d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:04.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.607+0000 7f1460a5e6c0 1 -- 10.20.193.27:0/1853101020 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f14440e38e0 con 0x7f145c115410 2026-02-13T02:25:04.611 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.607+0000 7f1460a5e6c0 1 -- 10.20.193.27:0/1853101020 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f14440a6360 con 0x7f145c115410 2026-02-13T02:25:04.612 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.607+0000 7f145affd6c0 1 --2- 10.20.193.27:0/1853101020 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f14300b5680 0x7f14300d5b30 secure :-1 s=READY pgs=276 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f14500098c0 tx=0x7f145002f580 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:04.722 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.720+0000 7f1461a646c0 1 -- 10.20.193.27:0/1853101020 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f145c10f5d0 con 0x7f145c115410 2026-02-13T02:25:04.722 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.720+0000 7f1460a5e6c0 1 -- 10.20.193.27:0/1853101020 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f14440a9510 con 0x7f145c115410 2026-02-13T02:25:04.722 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:25:04.723 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:25:04.724 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.721+0000 7f1461a646c0 1 -- 10.20.193.27:0/1853101020 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f14300b5680 msgr2=0x7f14300d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:04.724 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.721+0000 7f1461a646c0 1 --2- 10.20.193.27:0/1853101020 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f14300b5680 0x7f14300d5b30 secure :-1 s=READY pgs=276 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f14500098c0 tx=0x7f145002f580 comp rx=0 tx=0).stop 2026-02-13T02:25:04.724 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.721+0000 7f1461a646c0 1 -- 10.20.193.27:0/1853101020 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f145c115410 msgr2=0x7f145c11d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:04.724 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.721+0000 7f1461a646c0 1 --2- 10.20.193.27:0/1853101020 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f145c115410 0x7f145c11d830 secure :-1 s=READY pgs=425 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f144400b6c0 tx=0x7f144400bb90 comp rx=0 tx=0).stop 2026-02-13T02:25:04.724 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.721+0000 7f1461a646c0 1 -- 10.20.193.27:0/1853101020 shutdown_connections 2026-02-13T02:25:04.724 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.721+0000 7f1461a646c0 1 --2- 10.20.193.27:0/1853101020 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f14300b5680 0x7f14300d5b30 unknown :-1 s=CLOSED pgs=276 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:04.725 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.721+0000 7f1461a646c0 1 --2- 10.20.193.27:0/1853101020 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f145c122bf0 0x7f145c123090 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:04.725 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.721+0000 7f1461a646c0 1 --2- 10.20.193.27:0/1853101020 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f145c115410 0x7f145c11d830 unknown :-1 s=CLOSED pgs=425 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:04.725 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.721+0000 7f1461a646c0 1 --2- 10.20.193.27:0/1853101020 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f145c10f200 0x7f145c11d2f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:04.725 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.721+0000 7f1461a646c0 1 -- 10.20.193.27:0/1853101020 >> 10.20.193.27:0/1853101020 conn(0x7f145c0ecb00 msgr2=0x7f145c10cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:04.725 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.721+0000 7f1461a646c0 1 -- 10.20.193.27:0/1853101020 shutdown_connections 2026-02-13T02:25:04.725 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:04.721+0000 7f1461a646c0 1 -- 10.20.193.27:0/1853101020 wait complete. 2026-02-13T02:25:05.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:05 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1853101020' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:05.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:05 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1853101020' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:05.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:05 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1853101020' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:06.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:06 trial027 ceph-mon[24567]: pgmap v272: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:06.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:06 trial028 ceph-mon[26701]: pgmap v272: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:06.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:06 trial107 ceph-mon[27422]: pgmap v272: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:07.764 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:25:07.883 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:25:08.004 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.000+0000 7f39dc1d36c0 1 -- 10.20.193.27:0/1414626835 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f39d410fc20 msgr2=0x7f39d4114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:08.004 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.000+0000 7f39dc1d36c0 1 --2- 10.20.193.27:0/1414626835 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f39d410fc20 0x7f39d4114c60 secure :-1 s=READY pgs=426 gs=1 cs=0 l=1 c_cookie=10315754376787144007 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f39c80099a0 tx=0x7f39c802ed20 comp rx=0 tx=0).stop 2026-02-13T02:25:08.004 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.000+0000 7f39dc1d36c0 1 -- 10.20.193.27:0/1414626835 shutdown_connections 2026-02-13T02:25:08.004 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.000+0000 7f39dc1d36c0 1 --2- 10.20.193.27:0/1414626835 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f39d4115440 0x7f39d4175de0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:08.004 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.000+0000 7f39dc1d36c0 1 --2- 10.20.193.27:0/1414626835 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f39d410fc20 0x7f39d4114c60 unknown :-1 s=CLOSED pgs=426 gs=1 cs=0 l=1 c_cookie=10315754376787144007 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:08.005 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.000+0000 7f39dc1d36c0 1 --2- 10.20.193.27:0/1414626835 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f39d410f230 0x7f39d410f600 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:08.005 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.000+0000 7f39dc1d36c0 1 -- 10.20.193.27:0/1414626835 >> 10.20.193.27:0/1414626835 conn(0x7f39d40ecb30 msgr2=0x7f39d410cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:08.005 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.000+0000 7f39dc1d36c0 1 -- 10.20.193.27:0/1414626835 shutdown_connections 2026-02-13T02:25:08.005 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.000+0000 7f39dc1d36c0 1 -- 10.20.193.27:0/1414626835 wait complete. 2026-02-13T02:25:08.005 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.000+0000 7f39dc1d36c0 1 Processor -- start 2026-02-13T02:25:08.005 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.000+0000 7f39dc1d36c0 1 -- start start 2026-02-13T02:25:08.005 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.001+0000 7f39dc1d36c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f39d410f230 0x7f39d40f4d60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:08.005 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.001+0000 7f39dc1d36c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f39d4115440 0x7f39d4175ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:08.005 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.001+0000 7f39dc1d36c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f39d40f5390 0x7f39d40fa660 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:08.005 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.001+0000 7f39dc1d36c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f39d40f5a80 con 0x7f39d4115440 2026-02-13T02:25:08.005 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.001+0000 7f39dc1d36c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f39d40faba0 con 0x7f39d40f5390 2026-02-13T02:25:08.006 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.001+0000 7f39dc1d36c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f39d40fad60 con 0x7f39d410f230 2026-02-13T02:25:08.006 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.001+0000 7f39d97436c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f39d4115440 0x7f39d4175ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:08.006 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.001+0000 7f39d97436c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f39d4115440 0x7f39d4175ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:46938/0 (socket says 10.20.193.27:46938) 2026-02-13T02:25:08.006 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.001+0000 7f39d97436c0 1 -- 10.20.193.27:0/240164436 learned_addr learned my addr 10.20.193.27:0/240164436 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:25:08.006 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.001+0000 7f39da7456c0 1 --2- 10.20.193.27:0/240164436 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f39d40f5390 0x7f39d40fa660 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:08.006 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.001+0000 7f39d9f446c0 1 --2- 10.20.193.27:0/240164436 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f39d410f230 0x7f39d40f4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:08.006 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.001+0000 7f39d97436c0 1 -- 10.20.193.27:0/240164436 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f39d410f230 msgr2=0x7f39d40f4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:08.006 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.001+0000 7f39d97436c0 1 --2- 10.20.193.27:0/240164436 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f39d410f230 0x7f39d40f4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:08.006 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.001+0000 7f39d97436c0 1 -- 10.20.193.27:0/240164436 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f39d40f5390 msgr2=0x7f39d40fa660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:08.006 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.001+0000 7f39d97436c0 1 --2- 10.20.193.27:0/240164436 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f39d40f5390 0x7f39d40fa660 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:08.006 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.001+0000 7f39d97436c0 1 -- 10.20.193.27:0/240164436 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c8009590 con 0x7f39d4115440 2026-02-13T02:25:08.007 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.001+0000 7f39d97436c0 1 --2- 10.20.193.27:0/240164436 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f39d4115440 0x7f39d4175ce0 secure :-1 s=READY pgs=427 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f39c80023c0 tx=0x7f39c802fbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:08.007 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.001+0000 7f39c2ffd6c0 1 -- 10.20.193.27:0/240164436 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f39c8002ae0 con 0x7f39d4115440 2026-02-13T02:25:08.007 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.001+0000 7f39dc1d36c0 1 -- 10.20.193.27:0/240164436 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d40fafe0 con 0x7f39d4115440 2026-02-13T02:25:08.007 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.001+0000 7f39c2ffd6c0 1 -- 10.20.193.27:0/240164436 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f39c8007680 con 0x7f39d4115440 2026-02-13T02:25:08.007 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.001+0000 7f39c2ffd6c0 1 -- 10.20.193.27:0/240164436 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f39c80374d0 con 0x7f39d4115440 2026-02-13T02:25:08.007 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.001+0000 7f39dc1d36c0 1 -- 10.20.193.27:0/240164436 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d40fb460 con 0x7f39d4115440 2026-02-13T02:25:08.007 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.001+0000 7f39dc1d36c0 1 -- 10.20.193.27:0/240164436 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39d4110500 con 0x7f39d4115440 2026-02-13T02:25:08.007 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.003+0000 7f39c2ffd6c0 1 -- 10.20.193.27:0/240164436 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f39c8007cb0 con 0x7f39d4115440 2026-02-13T02:25:08.008 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.003+0000 7f39c2ffd6c0 1 --2- 10.20.193.27:0/240164436 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f39b00b5600 0x7f39b00d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:08.008 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.003+0000 7f39d9f446c0 1 --2- 10.20.193.27:0/240164436 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f39b00b5600 0x7f39b00d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:08.008 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.003+0000 7f39c2ffd6c0 1 -- 10.20.193.27:0/240164436 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f39c8107be0 con 0x7f39d4115440 2026-02-13T02:25:08.008 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.004+0000 7f39c2ffd6c0 1 -- 10.20.193.27:0/240164436 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f39c80ca660 con 0x7f39d4115440 2026-02-13T02:25:08.008 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.004+0000 7f39d9f446c0 1 --2- 10.20.193.27:0/240164436 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f39b00b5600 0x7f39b00d5ab0 secure :-1 s=READY pgs=277 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f39c4004750 tx=0x7f39c4006da0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:08.119 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.118+0000 7f39dc1d36c0 1 -- 10.20.193.27:0/240164436 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f39d410f600 con 0x7f39d4115440 2026-02-13T02:25:08.119 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.118+0000 7f39c2ffd6c0 1 -- 10.20.193.27:0/240164436 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f39c80cd810 con 0x7f39d4115440 2026-02-13T02:25:08.120 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:25:08.120 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:25:08.121 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.119+0000 7f39dc1d36c0 1 -- 10.20.193.27:0/240164436 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f39b00b5600 msgr2=0x7f39b00d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:08.121 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.119+0000 7f39dc1d36c0 1 --2- 10.20.193.27:0/240164436 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f39b00b5600 0x7f39b00d5ab0 secure :-1 s=READY pgs=277 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f39c4004750 tx=0x7f39c4006da0 comp rx=0 tx=0).stop 2026-02-13T02:25:08.121 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.119+0000 7f39dc1d36c0 1 -- 10.20.193.27:0/240164436 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f39d4115440 msgr2=0x7f39d4175ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:08.121 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.119+0000 7f39dc1d36c0 1 --2- 10.20.193.27:0/240164436 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f39d4115440 0x7f39d4175ce0 secure :-1 s=READY pgs=427 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f39c80023c0 tx=0x7f39c802fbf0 comp rx=0 tx=0).stop 2026-02-13T02:25:08.121 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.119+0000 7f39dc1d36c0 1 -- 10.20.193.27:0/240164436 shutdown_connections 2026-02-13T02:25:08.121 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.119+0000 7f39dc1d36c0 1 --2- 10.20.193.27:0/240164436 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f39b00b5600 0x7f39b00d5ab0 unknown :-1 s=CLOSED pgs=277 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:08.121 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.119+0000 7f39dc1d36c0 1 --2- 10.20.193.27:0/240164436 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f39d40f5390 0x7f39d40fa660 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:08.122 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.119+0000 7f39dc1d36c0 1 --2- 10.20.193.27:0/240164436 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f39d4115440 0x7f39d4175ce0 unknown :-1 s=CLOSED pgs=427 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:08.122 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.119+0000 7f39dc1d36c0 1 --2- 10.20.193.27:0/240164436 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f39d410f230 0x7f39d40f4d60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:08.122 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.119+0000 7f39dc1d36c0 1 -- 10.20.193.27:0/240164436 >> 10.20.193.27:0/240164436 conn(0x7f39d40ecb30 msgr2=0x7f39d410da30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:08.122 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.119+0000 7f39dc1d36c0 1 -- 10.20.193.27:0/240164436 shutdown_connections 2026-02-13T02:25:08.122 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:08.119+0000 7f39dc1d36c0 1 -- 10.20.193.27:0/240164436 wait complete. 2026-02-13T02:25:08.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:08 trial027 ceph-mon[24567]: pgmap v273: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:08.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:08 trial028 ceph-mon[26701]: pgmap v273: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:08.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:08 trial107 ceph-mon[27422]: pgmap v273: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:09.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:09 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/240164436' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:09.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:09 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/240164436' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:09.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:09 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/240164436' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:10.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:10 trial027 ceph-mon[24567]: pgmap v274: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:10.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:10 trial028 ceph-mon[26701]: pgmap v274: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:10.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:10 trial107 ceph-mon[27422]: pgmap v274: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:11.166 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:25:11.285 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:25:11.400 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.397+0000 7f004d8706c0 1 -- 10.20.193.27:0/2880657584 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f004810fc20 msgr2=0x7f0048114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:11.401 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.397+0000 7f004d8706c0 1 --2- 10.20.193.27:0/2880657584 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f004810fc20 0x7f0048114c60 secure :-1 s=READY pgs=428 gs=1 cs=0 l=1 c_cookie=852138629539803735 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f00300099a0 tx=0x7f003002ed20 comp rx=0 tx=0).stop 2026-02-13T02:25:11.401 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.397+0000 7f004d8706c0 1 -- 10.20.193.27:0/2880657584 shutdown_connections 2026-02-13T02:25:11.401 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.397+0000 7f004d8706c0 1 --2- 10.20.193.27:0/2880657584 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f0048115440 0x7f0048175de0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:11.401 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.397+0000 7f004d8706c0 1 --2- 10.20.193.27:0/2880657584 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f004810fc20 0x7f0048114c60 unknown :-1 s=CLOSED pgs=428 gs=1 cs=0 l=1 c_cookie=852138629539803735 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:11.401 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.397+0000 7f004d8706c0 1 --2- 10.20.193.27:0/2880657584 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f004810f230 0x7f004810f600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:11.401 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.397+0000 7f004d8706c0 1 -- 10.20.193.27:0/2880657584 >> 10.20.193.27:0/2880657584 conn(0x7f00480ecb30 msgr2=0x7f004810cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:11.401 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.397+0000 7f004d8706c0 1 -- 10.20.193.27:0/2880657584 shutdown_connections 2026-02-13T02:25:11.401 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.397+0000 7f004d8706c0 1 -- 10.20.193.27:0/2880657584 wait complete. 2026-02-13T02:25:11.401 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.397+0000 7f004d8706c0 1 Processor -- start 2026-02-13T02:25:11.402 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.397+0000 7f004d8706c0 1 -- start start 2026-02-13T02:25:11.402 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.397+0000 7f004d8706c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f004810f230 0x7f00480f4ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:11.402 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.397+0000 7f004d8706c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0048115440 0x7f0048175ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:11.402 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.397+0000 7f004d8706c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f00480f5310 0x7f00480fa590 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:11.402 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.397+0000 7f004d8706c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f00480f5a00 con 0x7f0048115440 2026-02-13T02:25:11.402 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.397+0000 7f004d8706c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f00480faad0 con 0x7f00480f5310 2026-02-13T02:25:11.402 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.397+0000 7f004d8706c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f00480fac90 con 0x7f004810f230 2026-02-13T02:25:11.402 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.397+0000 7f00467fc6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0048115440 0x7f0048175ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:11.402 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.397+0000 7f00467fc6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0048115440 0x7f0048175ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:46954/0 (socket says 10.20.193.27:46954) 2026-02-13T02:25:11.403 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.397+0000 7f00467fc6c0 1 -- 10.20.193.27:0/3393512609 learned_addr learned my addr 10.20.193.27:0/3393512609 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:25:11.403 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.398+0000 7f0046ffd6c0 1 --2- 10.20.193.27:0/3393512609 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f004810f230 0x7f00480f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:11.403 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.398+0000 7f00477fe6c0 1 --2- 10.20.193.27:0/3393512609 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f00480f5310 0x7f00480fa590 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:11.403 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.398+0000 7f00467fc6c0 1 -- 10.20.193.27:0/3393512609 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f004810f230 msgr2=0x7f00480f4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:11.403 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.398+0000 7f00467fc6c0 1 --2- 10.20.193.27:0/3393512609 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f004810f230 0x7f00480f4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:11.403 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.398+0000 7f00467fc6c0 1 -- 10.20.193.27:0/3393512609 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f00480f5310 msgr2=0x7f00480fa590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:11.404 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.398+0000 7f00467fc6c0 1 --2- 10.20.193.27:0/3393512609 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f00480f5310 0x7f00480fa590 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:11.404 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.398+0000 7f00467fc6c0 1 -- 10.20.193.27:0/3393512609 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0030009590 con 0x7f0048115440 2026-02-13T02:25:11.404 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.398+0000 7f00467fc6c0 1 --2- 10.20.193.27:0/3393512609 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0048115440 0x7f0048175ce0 secure :-1 s=READY pgs=429 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f00300023c0 tx=0x7f003002fbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:11.404 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.398+0000 7f004c86e6c0 1 -- 10.20.193.27:0/3393512609 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0030002ae0 con 0x7f0048115440 2026-02-13T02:25:11.404 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.398+0000 7f004c86e6c0 1 -- 10.20.193.27:0/3393512609 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f0030002c40 con 0x7f0048115440 2026-02-13T02:25:11.404 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.398+0000 7f004d8706c0 1 -- 10.20.193.27:0/3393512609 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00480faf10 con 0x7f0048115440 2026-02-13T02:25:11.405 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.398+0000 7f004c86e6c0 1 -- 10.20.193.27:0/3393512609 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f00300372f0 con 0x7f0048115440 2026-02-13T02:25:11.405 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.398+0000 7f004d8706c0 1 -- 10.20.193.27:0/3393512609 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00480fb3e0 con 0x7f0048115440 2026-02-13T02:25:11.405 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.398+0000 7f004d8706c0 1 -- 10.20.193.27:0/3393512609 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0048110500 con 0x7f0048115440 2026-02-13T02:25:11.406 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.400+0000 7f004c86e6c0 1 -- 10.20.193.27:0/3393512609 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f0030007810 con 0x7f0048115440 2026-02-13T02:25:11.406 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.400+0000 7f004c86e6c0 1 --2- 10.20.193.27:0/3393512609 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f001c0b5680 0x7f001c0d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:11.406 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.400+0000 7f004c86e6c0 1 -- 10.20.193.27:0/3393512609 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f0030107b30 con 0x7f0048115440 2026-02-13T02:25:11.406 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.400+0000 7f0046ffd6c0 1 --2- 10.20.193.27:0/3393512609 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f001c0b5680 0x7f001c0d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:11.407 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.400+0000 7f0046ffd6c0 1 --2- 10.20.193.27:0/3393512609 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f001c0b5680 0x7f001c0d5b30 secure :-1 s=READY pgs=278 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f003c005980 tx=0x7f003c005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:11.407 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.401+0000 7f004c86e6c0 1 -- 10.20.193.27:0/3393512609 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f00300ca5f0 con 0x7f0048115440 2026-02-13T02:25:11.519 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.518+0000 7f004d8706c0 1 -- 10.20.193.27:0/3393512609 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f004810f600 con 0x7f0048115440 2026-02-13T02:25:11.519 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.518+0000 7f004c86e6c0 1 -- 10.20.193.27:0/3393512609 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f0030007e60 con 0x7f0048115440 2026-02-13T02:25:11.520 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:25:11.520 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:25:11.521 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.519+0000 7f004d8706c0 1 -- 10.20.193.27:0/3393512609 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f001c0b5680 msgr2=0x7f001c0d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:11.521 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.519+0000 7f004d8706c0 1 --2- 10.20.193.27:0/3393512609 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f001c0b5680 0x7f001c0d5b30 secure :-1 s=READY pgs=278 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f003c005980 tx=0x7f003c005910 comp rx=0 tx=0).stop 2026-02-13T02:25:11.521 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.519+0000 7f004d8706c0 1 -- 10.20.193.27:0/3393512609 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0048115440 msgr2=0x7f0048175ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:11.521 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.519+0000 7f004d8706c0 1 --2- 10.20.193.27:0/3393512609 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0048115440 0x7f0048175ce0 secure :-1 s=READY pgs=429 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f00300023c0 tx=0x7f003002fbf0 comp rx=0 tx=0).stop 2026-02-13T02:25:11.521 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.519+0000 7f004d8706c0 1 -- 10.20.193.27:0/3393512609 shutdown_connections 2026-02-13T02:25:11.521 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.519+0000 7f004d8706c0 1 --2- 10.20.193.27:0/3393512609 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f001c0b5680 0x7f001c0d5b30 unknown :-1 s=CLOSED pgs=278 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:11.521 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.519+0000 7f004d8706c0 1 --2- 10.20.193.27:0/3393512609 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f00480f5310 0x7f00480fa590 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:11.522 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.519+0000 7f004d8706c0 1 --2- 10.20.193.27:0/3393512609 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0048115440 0x7f0048175ce0 unknown :-1 s=CLOSED pgs=429 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:11.522 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.519+0000 7f004d8706c0 1 --2- 10.20.193.27:0/3393512609 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f004810f230 0x7f00480f4ce0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:11.522 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.519+0000 7f004d8706c0 1 -- 10.20.193.27:0/3393512609 >> 10.20.193.27:0/3393512609 conn(0x7f00480ecb30 msgr2=0x7f004810d970 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:11.522 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.519+0000 7f004d8706c0 1 -- 10.20.193.27:0/3393512609 shutdown_connections 2026-02-13T02:25:11.522 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:11.519+0000 7f004d8706c0 1 -- 10.20.193.27:0/3393512609 wait complete. 2026-02-13T02:25:12.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:12 trial027 ceph-mon[24567]: pgmap v275: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:12.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:12 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3393512609' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:12.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:12 trial028 ceph-mon[26701]: pgmap v275: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:12.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:12 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3393512609' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:12.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:12 trial107 ceph-mon[27422]: pgmap v275: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:12.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:12 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3393512609' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:14.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:14 trial027 ceph-mon[24567]: pgmap v276: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:14.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:14 trial028 ceph-mon[26701]: pgmap v276: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:14.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:14 trial107 ceph-mon[27422]: pgmap v276: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:14.562 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:25:14.682 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:25:14.801 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.797+0000 7f83cd2146c0 1 -- 10.20.193.27:0/2296311989 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f83c8115440 msgr2=0x7f83c8175de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:14.801 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.797+0000 7f83cd2146c0 1 --2- 10.20.193.27:0/2296311989 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f83c8115440 0x7f83c8175de0 secure :-1 s=READY pgs=430 gs=1 cs=0 l=1 c_cookie=9879694441081669641 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f83b800aa70 tx=0x7f83b802fde0 comp rx=0 tx=0).stop 2026-02-13T02:25:14.801 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.797+0000 7f83cd2146c0 1 -- 10.20.193.27:0/2296311989 shutdown_connections 2026-02-13T02:25:14.802 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.797+0000 7f83cd2146c0 1 --2- 10.20.193.27:0/2296311989 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f83c8115440 0x7f83c8175de0 unknown :-1 s=CLOSED pgs=430 gs=1 cs=0 l=1 c_cookie=9879694441081669641 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:14.802 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.797+0000 7f83cd2146c0 1 --2- 10.20.193.27:0/2296311989 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f83c810fc20 0x7f83c8114c60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:14.802 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.797+0000 7f83cd2146c0 1 --2- 10.20.193.27:0/2296311989 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f83c810f230 0x7f83c810f600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:14.802 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.797+0000 7f83cd2146c0 1 -- 10.20.193.27:0/2296311989 >> 10.20.193.27:0/2296311989 conn(0x7f83c80ecb30 msgr2=0x7f83c810cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:14.802 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.797+0000 7f83cd2146c0 1 -- 10.20.193.27:0/2296311989 shutdown_connections 2026-02-13T02:25:14.802 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.797+0000 7f83cd2146c0 1 -- 10.20.193.27:0/2296311989 wait complete. 2026-02-13T02:25:14.802 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.797+0000 7f83cd2146c0 1 Processor -- start 2026-02-13T02:25:14.802 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.797+0000 7f83cd2146c0 1 -- start start 2026-02-13T02:25:14.802 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.797+0000 7f83cd2146c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f83c810f230 0x7f83c811d1e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:14.802 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.797+0000 7f83cd2146c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f83c8115440 0x7f83c811d720 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:14.803 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.797+0000 7f83cd2146c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f83c8122ae0 0x7f83c8122f80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:14.803 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.797+0000 7f83cd2146c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f83c81234f0 con 0x7f83c810f230 2026-02-13T02:25:14.803 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.797+0000 7f83cd2146c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f83c8123660 con 0x7f83c8122ae0 2026-02-13T02:25:14.803 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.797+0000 7f83cd2146c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f83c8123820 con 0x7f83c8115440 2026-02-13T02:25:14.803 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.797+0000 7f83c6d766c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f83c810f230 0x7f83c811d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:14.803 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.798+0000 7f83c6d766c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f83c810f230 0x7f83c811d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:46970/0 (socket says 10.20.193.27:46970) 2026-02-13T02:25:14.803 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.798+0000 7f83c6d766c0 1 -- 10.20.193.27:0/2141049912 learned_addr learned my addr 10.20.193.27:0/2141049912 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:25:14.803 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.798+0000 7f83c65756c0 1 --2- 10.20.193.27:0/2141049912 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f83c8115440 0x7f83c811d720 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:14.803 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.798+0000 7f83c6d766c0 1 -- 10.20.193.27:0/2141049912 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f83c8115440 msgr2=0x7f83c811d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:14.803 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.798+0000 7f83c6d766c0 1 --2- 10.20.193.27:0/2141049912 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f83c8115440 0x7f83c811d720 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:14.803 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.798+0000 7f83c6d766c0 1 -- 10.20.193.27:0/2141049912 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f83c8122ae0 msgr2=0x7f83c8122f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:14.803 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.798+0000 7f83c6d766c0 1 --2- 10.20.193.27:0/2141049912 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f83c8122ae0 0x7f83c8122f80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:14.804 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.798+0000 7f83c6d766c0 1 -- 10.20.193.27:0/2141049912 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b800a660 con 0x7f83c810f230 2026-02-13T02:25:14.804 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.798+0000 7f83c65756c0 1 --2- 10.20.193.27:0/2141049912 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f83c8115440 0x7f83c811d720 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:25:14.804 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.798+0000 7f83c6d766c0 1 --2- 10.20.193.27:0/2141049912 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f83c810f230 0x7f83c811d1e0 secure :-1 s=READY pgs=431 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f83bc009490 tx=0x7f83bc00bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:14.804 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.798+0000 7f83a7fff6c0 1 -- 10.20.193.27:0/2141049912 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f83bc004a80 con 0x7f83c810f230 2026-02-13T02:25:14.804 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.798+0000 7f83a7fff6c0 1 -- 10.20.193.27:0/2141049912 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f83bc009030 con 0x7f83c810f230 2026-02-13T02:25:14.804 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.798+0000 7f83cd2146c0 1 -- 10.20.193.27:0/2141049912 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83c8163cd0 con 0x7f83c810f230 2026-02-13T02:25:14.804 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.798+0000 7f83a7fff6c0 1 -- 10.20.193.27:0/2141049912 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f83bc00ce30 con 0x7f83c810f230 2026-02-13T02:25:14.804 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.798+0000 7f83cd2146c0 1 -- 10.20.193.27:0/2141049912 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83c8164160 con 0x7f83c810f230 2026-02-13T02:25:14.804 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.798+0000 7f83cd2146c0 1 -- 10.20.193.27:0/2141049912 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83c8110500 con 0x7f83c810f230 2026-02-13T02:25:14.805 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.800+0000 7f83a7fff6c0 1 -- 10.20.193.27:0/2141049912 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f83bc004060 con 0x7f83c810f230 2026-02-13T02:25:14.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.800+0000 7f83a7fff6c0 1 --2- 10.20.193.27:0/2141049912 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f839c0b5600 0x7f839c0d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:14.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.801+0000 7f83c65756c0 1 --2- 10.20.193.27:0/2141049912 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f839c0b5600 0x7f839c0d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:14.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.801+0000 7f83a7fff6c0 1 -- 10.20.193.27:0/2141049912 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f83bc0e3b60 con 0x7f83c810f230 2026-02-13T02:25:14.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.801+0000 7f83a7fff6c0 1 -- 10.20.193.27:0/2141049912 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f83bc0a95b0 con 0x7f83c810f230 2026-02-13T02:25:14.806 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.801+0000 7f83c65756c0 1 --2- 10.20.193.27:0/2141049912 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f839c0b5600 0x7f839c0d5ab0 secure :-1 s=READY pgs=279 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f83c8123b70 tx=0x7f83b0005f10 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:14.920 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.917+0000 7f83cd2146c0 1 -- 10.20.193.27:0/2141049912 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f83c810f600 con 0x7f83c810f230 2026-02-13T02:25:14.920 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.917+0000 7f83a7fff6c0 1 -- 10.20.193.27:0/2141049912 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f83bc0a9790 con 0x7f83c810f230 2026-02-13T02:25:14.920 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:25:14.921 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:25:14.922 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.920+0000 7f83cd2146c0 1 -- 10.20.193.27:0/2141049912 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f839c0b5600 msgr2=0x7f839c0d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:14.923 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.920+0000 7f83cd2146c0 1 --2- 10.20.193.27:0/2141049912 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f839c0b5600 0x7f839c0d5ab0 secure :-1 s=READY pgs=279 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f83c8123b70 tx=0x7f83b0005f10 comp rx=0 tx=0).stop 2026-02-13T02:25:14.923 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.920+0000 7f83cd2146c0 1 -- 10.20.193.27:0/2141049912 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f83c810f230 msgr2=0x7f83c811d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:14.923 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.920+0000 7f83cd2146c0 1 --2- 10.20.193.27:0/2141049912 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f83c810f230 0x7f83c811d1e0 secure :-1 s=READY pgs=431 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f83bc009490 tx=0x7f83bc00bca0 comp rx=0 tx=0).stop 2026-02-13T02:25:14.923 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.920+0000 7f83cd2146c0 1 -- 10.20.193.27:0/2141049912 shutdown_connections 2026-02-13T02:25:14.923 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.920+0000 7f83cd2146c0 1 --2- 10.20.193.27:0/2141049912 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f839c0b5600 0x7f839c0d5ab0 unknown :-1 s=CLOSED pgs=279 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:14.923 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.920+0000 7f83cd2146c0 1 --2- 10.20.193.27:0/2141049912 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f83c8122ae0 0x7f83c8122f80 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:14.923 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.920+0000 7f83cd2146c0 1 --2- 10.20.193.27:0/2141049912 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f83c8115440 0x7f83c811d720 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:14.923 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.920+0000 7f83cd2146c0 1 --2- 10.20.193.27:0/2141049912 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f83c810f230 0x7f83c811d1e0 unknown :-1 s=CLOSED pgs=431 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:14.923 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.920+0000 7f83cd2146c0 1 -- 10.20.193.27:0/2141049912 >> 10.20.193.27:0/2141049912 conn(0x7f83c80ecb30 msgr2=0x7f83c810cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:14.923 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.920+0000 7f83cd2146c0 1 -- 10.20.193.27:0/2141049912 shutdown_connections 2026-02-13T02:25:14.923 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:14.920+0000 7f83cd2146c0 1 -- 10.20.193.27:0/2141049912 wait complete. 2026-02-13T02:25:15.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:15 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2141049912' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:15.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:15 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2141049912' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:15.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:15 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2141049912' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:16.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:16 trial027 ceph-mon[24567]: pgmap v277: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:16.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:16 trial028 ceph-mon[26701]: pgmap v277: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:16.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:16 trial107 ceph-mon[27422]: pgmap v277: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:17.966 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:25:18.088 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:25:18.200 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.198+0000 7f9a92b056c0 1 -- 10.20.193.27:0/3376300478 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9a8c10cfb0 msgr2=0x7f9a8c10d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:18.200 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.198+0000 7f9a92b056c0 1 --2- 10.20.193.27:0/3376300478 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9a8c10cfb0 0x7f9a8c10d380 secure :-1 s=READY pgs=432 gs=1 cs=0 l=1 c_cookie=4496023583214692055 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a7c0099a0 tx=0x7f9a7c02ed20 comp rx=0 tx=0).stop 2026-02-13T02:25:18.200 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.198+0000 7f9a92b056c0 1 -- 10.20.193.27:0/3376300478 shutdown_connections 2026-02-13T02:25:18.201 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.198+0000 7f9a92b056c0 1 --2- 10.20.193.27:0/3376300478 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9a8c1131c0 0x7f9a8c173b60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:18.201 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.198+0000 7f9a92b056c0 1 --2- 10.20.193.27:0/3376300478 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a8c10d9a0 0x7f9a8c1129e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:18.201 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.198+0000 7f9a92b056c0 1 --2- 10.20.193.27:0/3376300478 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9a8c10cfb0 0x7f9a8c10d380 unknown :-1 s=CLOSED pgs=432 gs=1 cs=0 l=1 c_cookie=4496023583214692055 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:18.201 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.198+0000 7f9a92b056c0 1 -- 10.20.193.27:0/3376300478 >> 10.20.193.27:0/3376300478 conn(0x7f9a8c0ea8b0 msgr2=0x7f9a8c10acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:18.201 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.198+0000 7f9a92b056c0 1 -- 10.20.193.27:0/3376300478 shutdown_connections 2026-02-13T02:25:18.201 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.198+0000 7f9a92b056c0 1 -- 10.20.193.27:0/3376300478 wait complete. 2026-02-13T02:25:18.203 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.198+0000 7f9a92b056c0 1 Processor -- start 2026-02-13T02:25:18.203 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.198+0000 7f9a92b056c0 1 -- start start 2026-02-13T02:25:18.204 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.198+0000 7f9a92b056c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9a8c10cfb0 0x7f9a8c11b090 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:18.204 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.198+0000 7f9a92b056c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9a8c1131c0 0x7f9a8c11b5d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:18.204 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.198+0000 7f9a92b056c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a8c120990 0x7f9a8c120e30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:18.204 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.198+0000 7f9a92b056c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c1213a0 con 0x7f9a8c10cfb0 2026-02-13T02:25:18.204 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.198+0000 7f9a92b056c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c121510 con 0x7f9a8c120990 2026-02-13T02:25:18.204 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.198+0000 7f9a92b056c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c1216d0 con 0x7f9a8c1131c0 2026-02-13T02:25:18.204 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.198+0000 7f9a908766c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9a8c10cfb0 0x7f9a8c11b090 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:18.204 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.199+0000 7f9a908766c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9a8c10cfb0 0x7f9a8c11b090 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:39878/0 (socket says 10.20.193.27:39878) 2026-02-13T02:25:18.204 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.199+0000 7f9a908766c0 1 -- 10.20.193.27:0/991261535 learned_addr learned my addr 10.20.193.27:0/991261535 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:25:18.204 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.199+0000 7f9a910776c0 1 --2- 10.20.193.27:0/991261535 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a8c120990 0x7f9a8c120e30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:18.204 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.199+0000 7f9a908766c0 1 -- 10.20.193.27:0/991261535 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9a8c1131c0 msgr2=0x7f9a8c11b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:18.205 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.199+0000 7f9a908766c0 1 --2- 10.20.193.27:0/991261535 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9a8c1131c0 0x7f9a8c11b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:18.205 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.199+0000 7f9a908766c0 1 -- 10.20.193.27:0/991261535 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a8c120990 msgr2=0x7f9a8c120e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:18.205 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.199+0000 7f9a908766c0 1 --2- 10.20.193.27:0/991261535 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a8c120990 0x7f9a8c120e30 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:18.205 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.199+0000 7f9a908766c0 1 -- 10.20.193.27:0/991261535 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a7c009590 con 0x7f9a8c10cfb0 2026-02-13T02:25:18.205 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.199+0000 7f9a908766c0 1 --2- 10.20.193.27:0/991261535 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9a8c10cfb0 0x7f9a8c11b090 secure :-1 s=READY pgs=433 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a7c02f230 tx=0x7f9a7c02fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:18.205 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.199+0000 7f9a81ffb6c0 1 -- 10.20.193.27:0/991261535 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9a7c002ba0 con 0x7f9a8c10cfb0 2026-02-13T02:25:18.205 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.199+0000 7f9a92b056c0 1 -- 10.20.193.27:0/991261535 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a8c161b20 con 0x7f9a8c10cfb0 2026-02-13T02:25:18.205 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.199+0000 7f9a81ffb6c0 1 -- 10.20.193.27:0/991261535 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9a7c002d00 con 0x7f9a8c10cfb0 2026-02-13T02:25:18.205 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.199+0000 7f9a81ffb6c0 1 -- 10.20.193.27:0/991261535 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9a7c0372f0 con 0x7f9a8c10cfb0 2026-02-13T02:25:18.205 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.199+0000 7f9a92b056c0 1 -- 10.20.193.27:0/991261535 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a8c161f50 con 0x7f9a8c10cfb0 2026-02-13T02:25:18.205 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.199+0000 7f9a92b056c0 1 -- 10.20.193.27:0/991261535 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a8c10e280 con 0x7f9a8c10cfb0 2026-02-13T02:25:18.205 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.201+0000 7f9a81ffb6c0 1 -- 10.20.193.27:0/991261535 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f9a7c0078d0 con 0x7f9a8c10cfb0 2026-02-13T02:25:18.207 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.202+0000 7f9a81ffb6c0 1 --2- 10.20.193.27:0/991261535 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9a680b5600 0x7f9a680d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:18.207 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.202+0000 7f9a83fff6c0 1 --2- 10.20.193.27:0/991261535 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9a680b5600 0x7f9a680d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:18.207 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.202+0000 7f9a81ffb6c0 1 -- 10.20.193.27:0/991261535 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f9a7c107c20 con 0x7f9a8c10cfb0 2026-02-13T02:25:18.207 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.202+0000 7f9a81ffb6c0 1 -- 10.20.193.27:0/991261535 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f9a7c140680 con 0x7f9a8c10cfb0 2026-02-13T02:25:18.207 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.202+0000 7f9a83fff6c0 1 --2- 10.20.193.27:0/991261535 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9a680b5600 0x7f9a680d5ab0 secure :-1 s=READY pgs=280 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a74005980 tx=0x7f9a74005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:18.318 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:18 trial027 ceph-mon[24567]: pgmap v278: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:18.319 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.316+0000 7f9a92b056c0 1 -- 10.20.193.27:0/991261535 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f9a8c10d9a0 con 0x7f9a8c10cfb0 2026-02-13T02:25:18.319 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.316+0000 7f9a81ffb6c0 1 -- 10.20.193.27:0/991261535 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f9a7c0ca6a0 con 0x7f9a8c10cfb0 2026-02-13T02:25:18.319 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:25:18.319 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:25:18.320 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.317+0000 7f9a92b056c0 1 -- 10.20.193.27:0/991261535 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9a680b5600 msgr2=0x7f9a680d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:18.320 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.317+0000 7f9a92b056c0 1 --2- 10.20.193.27:0/991261535 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9a680b5600 0x7f9a680d5ab0 secure :-1 s=READY pgs=280 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a74005980 tx=0x7f9a74005910 comp rx=0 tx=0).stop 2026-02-13T02:25:18.320 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.317+0000 7f9a92b056c0 1 -- 10.20.193.27:0/991261535 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9a8c10cfb0 msgr2=0x7f9a8c11b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:18.320 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.317+0000 7f9a92b056c0 1 --2- 10.20.193.27:0/991261535 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9a8c10cfb0 0x7f9a8c11b090 secure :-1 s=READY pgs=433 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a7c02f230 tx=0x7f9a7c02fd40 comp rx=0 tx=0).stop 2026-02-13T02:25:18.321 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.317+0000 7f9a92b056c0 1 -- 10.20.193.27:0/991261535 shutdown_connections 2026-02-13T02:25:18.321 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.317+0000 7f9a92b056c0 1 --2- 10.20.193.27:0/991261535 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9a680b5600 0x7f9a680d5ab0 unknown :-1 s=CLOSED pgs=280 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:18.321 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.317+0000 7f9a92b056c0 1 --2- 10.20.193.27:0/991261535 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a8c120990 0x7f9a8c120e30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:18.321 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.317+0000 7f9a92b056c0 1 --2- 10.20.193.27:0/991261535 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9a8c1131c0 0x7f9a8c11b5d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:18.321 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.317+0000 7f9a92b056c0 1 --2- 10.20.193.27:0/991261535 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9a8c10cfb0 0x7f9a8c11b090 unknown :-1 s=CLOSED pgs=433 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:18.321 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.317+0000 7f9a92b056c0 1 -- 10.20.193.27:0/991261535 >> 10.20.193.27:0/991261535 conn(0x7f9a8c0ea8b0 msgr2=0x7f9a8c10ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:18.321 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.317+0000 7f9a92b056c0 1 -- 10.20.193.27:0/991261535 shutdown_connections 2026-02-13T02:25:18.321 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:18.317+0000 7f9a92b056c0 1 -- 10.20.193.27:0/991261535 wait complete. 2026-02-13T02:25:18.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:18 trial028 ceph-mon[26701]: pgmap v278: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:18.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:18 trial107 ceph-mon[27422]: pgmap v278: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:19.314 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:19 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/991261535' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:19.315 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:19 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:25:19.315 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:19 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:25:19.315 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:19 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:25:19.315 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:19 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:25:19.315 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:19 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:25:19.315 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:19 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:25:19.315 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:19 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:25:19.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:19 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/991261535' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:19.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:19 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:25:19.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:19 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:25:19.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:19 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:25:19.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:19 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:25:19.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:19 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:25:19.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:19 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:25:19.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:19 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:25:19.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:19 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/991261535' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:19.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:19 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:25:19.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:19 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:25:19.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:19 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:25:19.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:19 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:25:19.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:19 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:25:19.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:19 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:25:19.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:19 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:25:20.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:20 trial027 ceph-mon[24567]: pgmap v279: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:20.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:20 trial027 ceph-mon[24567]: pgmap v280: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 981 B/s rd, 1 op/s 2026-02-13T02:25:20.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:20 trial027 ceph-mon[24567]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:25:20.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:20 trial027 ceph-mon[24567]: Cluster is now healthy 2026-02-13T02:25:20.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:20 trial028 ceph-mon[26701]: pgmap v279: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:20.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:20 trial028 ceph-mon[26701]: pgmap v280: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 981 B/s rd, 1 op/s 2026-02-13T02:25:20.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:20 trial028 ceph-mon[26701]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:25:20.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:20 trial028 ceph-mon[26701]: Cluster is now healthy 2026-02-13T02:25:20.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:20 trial107 ceph-mon[27422]: pgmap v279: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:20.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:20 trial107 ceph-mon[27422]: pgmap v280: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 981 B/s rd, 1 op/s 2026-02-13T02:25:20.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:20 trial107 ceph-mon[27422]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:25:20.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:20 trial107 ceph-mon[27422]: Cluster is now healthy 2026-02-13T02:25:21.362 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:25:21.384 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:21 trial027 ceph-mon[24567]: pgmap v281: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:25:21.384 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:21 trial027 ceph-mon[24567]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:25:21.481 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:25:21.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:21 trial028 ceph-mon[26701]: pgmap v281: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:25:21.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:21 trial028 ceph-mon[26701]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:25:21.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:21 trial107 ceph-mon[27422]: pgmap v281: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:25:21.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:21 trial107 ceph-mon[27422]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:25:21.599 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.595+0000 7f31ec4096c0 1 -- 10.20.193.27:0/1475131260 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f31e410cf80 msgr2=0x7f31e410d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:21.600 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.595+0000 7f31ec4096c0 1 --2- 10.20.193.27:0/1475131260 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f31e410cf80 0x7f31e410d350 secure :-1 s=READY pgs=434 gs=1 cs=0 l=1 c_cookie=8330075762260672106 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f31d40099a0 tx=0x7f31d402ed50 comp rx=0 tx=0).stop 2026-02-13T02:25:21.600 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.595+0000 7f31ec4096c0 1 -- 10.20.193.27:0/1475131260 shutdown_connections 2026-02-13T02:25:21.600 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.595+0000 7f31ec4096c0 1 --2- 10.20.193.27:0/1475131260 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f31e4113190 0x7f31e4173b30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:21.600 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.595+0000 7f31ec4096c0 1 --2- 10.20.193.27:0/1475131260 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f31e410d970 0x7f31e41129b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:21.600 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.595+0000 7f31ec4096c0 1 --2- 10.20.193.27:0/1475131260 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f31e410cf80 0x7f31e410d350 unknown :-1 s=CLOSED pgs=434 gs=1 cs=0 l=1 c_cookie=8330075762260672106 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:21.600 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.595+0000 7f31ec4096c0 1 -- 10.20.193.27:0/1475131260 >> 10.20.193.27:0/1475131260 conn(0x7f31e40ea880 msgr2=0x7f31e410ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:21.600 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.595+0000 7f31ec4096c0 1 -- 10.20.193.27:0/1475131260 shutdown_connections 2026-02-13T02:25:21.600 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.595+0000 7f31ec4096c0 1 -- 10.20.193.27:0/1475131260 wait complete. 2026-02-13T02:25:21.600 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.595+0000 7f31ec4096c0 1 Processor -- start 2026-02-13T02:25:21.600 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.595+0000 7f31ec4096c0 1 -- start start 2026-02-13T02:25:21.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.595+0000 7f31ec4096c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f31e410cf80 0x7f31e411b030 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:21.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.595+0000 7f31ec4096c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f31e4113190 0x7f31e411b570 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:21.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.595+0000 7f31ec4096c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f31e4120930 0x7f31e4120dd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:21.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.595+0000 7f31ec4096c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f31e4121310 con 0x7f31e410cf80 2026-02-13T02:25:21.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.595+0000 7f31ec4096c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f31e4121480 con 0x7f31e4120930 2026-02-13T02:25:21.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.595+0000 7f31ec4096c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f31e4121640 con 0x7f31e4113190 2026-02-13T02:25:21.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.595+0000 7f31ea17a6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f31e410cf80 0x7f31e411b030 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:21.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.595+0000 7f31ea17a6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f31e410cf80 0x7f31e411b030 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:39908/0 (socket says 10.20.193.27:39908) 2026-02-13T02:25:21.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.596+0000 7f31ea17a6c0 1 -- 10.20.193.27:0/1413442594 learned_addr learned my addr 10.20.193.27:0/1413442594 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:25:21.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.596+0000 7f31ea97b6c0 1 --2- 10.20.193.27:0/1413442594 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f31e4120930 0x7f31e4120dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:21.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.596+0000 7f31e99796c0 1 --2- 10.20.193.27:0/1413442594 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f31e4113190 0x7f31e411b570 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:21.601 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.596+0000 7f31ea17a6c0 1 -- 10.20.193.27:0/1413442594 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f31e4113190 msgr2=0x7f31e411b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:21.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.596+0000 7f31ea17a6c0 1 --2- 10.20.193.27:0/1413442594 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f31e4113190 0x7f31e411b570 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:21.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.596+0000 7f31ea17a6c0 1 -- 10.20.193.27:0/1413442594 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f31e4120930 msgr2=0x7f31e4120dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:21.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.596+0000 7f31ea17a6c0 1 --2- 10.20.193.27:0/1413442594 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f31e4120930 0x7f31e4120dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:21.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.596+0000 7f31ea17a6c0 1 -- 10.20.193.27:0/1413442594 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d4009590 con 0x7f31e410cf80 2026-02-13T02:25:21.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.596+0000 7f31ea97b6c0 1 --2- 10.20.193.27:0/1413442594 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f31e4120930 0x7f31e4120dd0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:25:21.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.596+0000 7f31ea17a6c0 1 --2- 10.20.193.27:0/1413442594 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f31e410cf80 0x7f31e411b030 secure :-1 s=READY pgs=435 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f31d40023e0 tx=0x7f31d402fd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:21.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.596+0000 7f31cb7fe6c0 1 -- 10.20.193.27:0/1413442594 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f31d4002ba0 con 0x7f31e410cf80 2026-02-13T02:25:21.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.596+0000 7f31cb7fe6c0 1 -- 10.20.193.27:0/1413442594 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f31d4002d00 con 0x7f31e410cf80 2026-02-13T02:25:21.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.596+0000 7f31ec4096c0 1 -- 10.20.193.27:0/1413442594 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31e4161ac0 con 0x7f31e410cf80 2026-02-13T02:25:21.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.596+0000 7f31cb7fe6c0 1 -- 10.20.193.27:0/1413442594 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f31d40362d0 con 0x7f31e410cf80 2026-02-13T02:25:21.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.596+0000 7f31ec4096c0 1 -- 10.20.193.27:0/1413442594 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31e4161f90 con 0x7f31e410cf80 2026-02-13T02:25:21.602 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.596+0000 7f31ec4096c0 1 -- 10.20.193.27:0/1413442594 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31e410e250 con 0x7f31e410cf80 2026-02-13T02:25:21.604 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.598+0000 7f31cb7fe6c0 1 -- 10.20.193.27:0/1413442594 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f31d40071c0 con 0x7f31e410cf80 2026-02-13T02:25:21.604 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.599+0000 7f31cb7fe6c0 1 --2- 10.20.193.27:0/1413442594 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f31c00b5680 0x7f31c00d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:21.604 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.599+0000 7f31e99796c0 1 --2- 10.20.193.27:0/1413442594 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f31c00b5680 0x7f31c00d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:21.604 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.599+0000 7f31cb7fe6c0 1 -- 10.20.193.27:0/1413442594 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f31d4107dc0 con 0x7f31e410cf80 2026-02-13T02:25:21.604 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.599+0000 7f31e99796c0 1 --2- 10.20.193.27:0/1413442594 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f31c00b5680 0x7f31c00d5b30 secure :-1 s=READY pgs=281 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f31e41219c0 tx=0x7f31d80094b0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:21.604 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.599+0000 7f31cb7fe6c0 1 -- 10.20.193.27:0/1413442594 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f31d40ca8d0 con 0x7f31e410cf80 2026-02-13T02:25:21.717 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.714+0000 7f31ec4096c0 1 -- 10.20.193.27:0/1413442594 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f31e411c180 con 0x7f31e410cf80 2026-02-13T02:25:21.717 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.714+0000 7f31cb7fe6c0 1 -- 10.20.193.27:0/1413442594 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f31d40cd9f0 con 0x7f31e410cf80 2026-02-13T02:25:21.717 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:25:21.717 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:25:21.718 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.715+0000 7f31ec4096c0 1 -- 10.20.193.27:0/1413442594 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f31c00b5680 msgr2=0x7f31c00d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:21.719 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.715+0000 7f31ec4096c0 1 --2- 10.20.193.27:0/1413442594 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f31c00b5680 0x7f31c00d5b30 secure :-1 s=READY pgs=281 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f31e41219c0 tx=0x7f31d80094b0 comp rx=0 tx=0).stop 2026-02-13T02:25:21.719 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.715+0000 7f31ec4096c0 1 -- 10.20.193.27:0/1413442594 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f31e410cf80 msgr2=0x7f31e411b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:21.719 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.715+0000 7f31ec4096c0 1 --2- 10.20.193.27:0/1413442594 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f31e410cf80 0x7f31e411b030 secure :-1 s=READY pgs=435 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f31d40023e0 tx=0x7f31d402fd30 comp rx=0 tx=0).stop 2026-02-13T02:25:21.719 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.715+0000 7f31ec4096c0 1 -- 10.20.193.27:0/1413442594 shutdown_connections 2026-02-13T02:25:21.719 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.715+0000 7f31ec4096c0 1 --2- 10.20.193.27:0/1413442594 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f31c00b5680 0x7f31c00d5b30 unknown :-1 s=CLOSED pgs=281 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:21.719 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.715+0000 7f31ec4096c0 1 --2- 10.20.193.27:0/1413442594 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f31e4120930 0x7f31e4120dd0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:21.719 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.715+0000 7f31ec4096c0 1 --2- 10.20.193.27:0/1413442594 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f31e4113190 0x7f31e411b570 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:21.719 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.715+0000 7f31ec4096c0 1 --2- 10.20.193.27:0/1413442594 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f31e410cf80 0x7f31e411b030 unknown :-1 s=CLOSED pgs=435 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:21.719 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.715+0000 7f31ec4096c0 1 -- 10.20.193.27:0/1413442594 >> 10.20.193.27:0/1413442594 conn(0x7f31e40ea880 msgr2=0x7f31e410ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:21.719 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.715+0000 7f31ec4096c0 1 -- 10.20.193.27:0/1413442594 shutdown_connections 2026-02-13T02:25:21.719 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:21.715+0000 7f31ec4096c0 1 -- 10.20.193.27:0/1413442594 wait complete. 2026-02-13T02:25:22.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:22 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1413442594' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:22.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:22 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1413442594' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:22.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:22 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1413442594' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:23.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:23 trial027 ceph-mon[24567]: pgmap v282: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 565 B/s rd, 1 op/s 2026-02-13T02:25:23.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:23 trial028 ceph-mon[26701]: pgmap v282: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 565 B/s rd, 1 op/s 2026-02-13T02:25:23.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:23 trial107 ceph-mon[27422]: pgmap v282: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 565 B/s rd, 1 op/s 2026-02-13T02:25:24.759 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:25:24.883 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:25:25.006 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.001+0000 7fa6a913b6c0 1 -- 10.20.193.27:0/2216598363 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa6a4115410 msgr2=0x7fa6a4175db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:25.006 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.001+0000 7fa6a913b6c0 1 --2- 10.20.193.27:0/2216598363 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa6a4115410 0x7fa6a4175db0 secure :-1 s=READY pgs=436 gs=1 cs=0 l=1 c_cookie=1405317085222759159 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa69400aa70 tx=0x7fa69402fde0 comp rx=0 tx=0).stop 2026-02-13T02:25:25.006 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.001+0000 7fa6a913b6c0 1 -- 10.20.193.27:0/2216598363 shutdown_connections 2026-02-13T02:25:25.006 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.001+0000 7fa6a913b6c0 1 --2- 10.20.193.27:0/2216598363 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa6a4115410 0x7fa6a4175db0 unknown :-1 s=CLOSED pgs=436 gs=1 cs=0 l=1 c_cookie=1405317085222759159 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:25.006 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.001+0000 7fa6a913b6c0 1 --2- 10.20.193.27:0/2216598363 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa6a410fbf0 0x7fa6a4114c30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:25.007 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.001+0000 7fa6a913b6c0 1 --2- 10.20.193.27:0/2216598363 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa6a410f200 0x7fa6a410f5d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:25.007 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.001+0000 7fa6a913b6c0 1 -- 10.20.193.27:0/2216598363 >> 10.20.193.27:0/2216598363 conn(0x7fa6a40ecb00 msgr2=0x7fa6a410cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:25.007 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.001+0000 7fa6a913b6c0 1 -- 10.20.193.27:0/2216598363 shutdown_connections 2026-02-13T02:25:25.007 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.001+0000 7fa6a913b6c0 1 -- 10.20.193.27:0/2216598363 wait complete. 2026-02-13T02:25:25.007 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.002+0000 7fa6a913b6c0 1 Processor -- start 2026-02-13T02:25:25.007 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.002+0000 7fa6a913b6c0 1 -- start start 2026-02-13T02:25:25.007 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.002+0000 7fa6a913b6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa6a410f200 0x7fa6a411d290 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:25.007 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.002+0000 7fa6a913b6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa6a4115410 0x7fa6a411d7d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:25.007 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.002+0000 7fa6a913b6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa6a4122b90 0x7fa6a4123030 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:25.007 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.002+0000 7fa6a913b6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a41235a0 con 0x7fa6a4115410 2026-02-13T02:25:25.007 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.002+0000 7fa6a913b6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a4123710 con 0x7fa6a4122b90 2026-02-13T02:25:25.008 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.002+0000 7fa6a913b6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a41238d0 con 0x7fa6a410f200 2026-02-13T02:25:25.008 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.002+0000 7fa6a25756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa6a4115410 0x7fa6a411d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:25.008 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.002+0000 7fa6a25756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa6a4115410 0x7fa6a411d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:39930/0 (socket says 10.20.193.27:39930) 2026-02-13T02:25:25.008 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.002+0000 7fa6a25756c0 1 -- 10.20.193.27:0/3737453861 learned_addr learned my addr 10.20.193.27:0/3737453861 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:25:25.008 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.002+0000 7fa6a2d766c0 1 --2- 10.20.193.27:0/3737453861 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa6a410f200 0x7fa6a411d290 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:25.008 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.002+0000 7fa6a35776c0 1 --2- 10.20.193.27:0/3737453861 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa6a4122b90 0x7fa6a4123030 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:25.008 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.002+0000 7fa6a25756c0 1 -- 10.20.193.27:0/3737453861 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa6a410f200 msgr2=0x7fa6a411d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:25.008 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.002+0000 7fa6a25756c0 1 --2- 10.20.193.27:0/3737453861 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa6a410f200 0x7fa6a411d290 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:25.008 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.002+0000 7fa6a25756c0 1 -- 10.20.193.27:0/3737453861 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa6a4122b90 msgr2=0x7fa6a4123030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:25.008 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.002+0000 7fa6a25756c0 1 --2- 10.20.193.27:0/3737453861 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa6a4122b90 0x7fa6a4123030 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:25.008 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.002+0000 7fa6a25756c0 1 -- 10.20.193.27:0/3737453861 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa69400a660 con 0x7fa6a4115410 2026-02-13T02:25:25.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.002+0000 7fa6a2d766c0 1 --2- 10.20.193.27:0/3737453861 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa6a410f200 0x7fa6a411d290 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:25:25.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.002+0000 7fa6a25756c0 1 --2- 10.20.193.27:0/3737453861 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa6a4115410 0x7fa6a411d7d0 secure :-1 s=READY pgs=437 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa68c005dc0 tx=0x7fa68c00bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:25.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.002+0000 7fa67bfff6c0 1 -- 10.20.193.27:0/3737453861 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa68c004250 con 0x7fa6a4115410 2026-02-13T02:25:25.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.002+0000 7fa6a913b6c0 1 -- 10.20.193.27:0/3737453861 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a4163d80 con 0x7fa6a4115410 2026-02-13T02:25:25.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.002+0000 7fa67bfff6c0 1 -- 10.20.193.27:0/3737453861 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa68c009070 con 0x7fa6a4115410 2026-02-13T02:25:25.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.002+0000 7fa67bfff6c0 1 -- 10.20.193.27:0/3737453861 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa68c00cd50 con 0x7fa6a4115410 2026-02-13T02:25:25.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.002+0000 7fa6a913b6c0 1 -- 10.20.193.27:0/3737453861 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a4164260 con 0x7fa6a4115410 2026-02-13T02:25:25.009 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.003+0000 7fa6a913b6c0 1 -- 10.20.193.27:0/3737453861 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6a41104d0 con 0x7fa6a4115410 2026-02-13T02:25:25.010 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.004+0000 7fa67bfff6c0 1 -- 10.20.193.27:0/3737453861 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7fa68c0028c0 con 0x7fa6a4115410 2026-02-13T02:25:25.010 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.005+0000 7fa67bfff6c0 1 --2- 10.20.193.27:0/3737453861 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa6740b5600 0x7fa6740d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:25.010 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.005+0000 7fa6a2d766c0 1 --2- 10.20.193.27:0/3737453861 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa6740b5600 0x7fa6740d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:25.010 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.005+0000 7fa67bfff6c0 1 -- 10.20.193.27:0/3737453861 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7fa68c0e3a80 con 0x7fa6a4115410 2026-02-13T02:25:25.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.005+0000 7fa6a2d766c0 1 --2- 10.20.193.27:0/3737453861 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa6740b5600 0x7fa6740d5ab0 secure :-1 s=READY pgs=282 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa698004750 tx=0x7fa698009630 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:25.011 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.006+0000 7fa67bfff6c0 1 -- 10.20.193.27:0/3737453861 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fa68c0a6540 con 0x7fa6a4115410 2026-02-13T02:25:25.122 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.120+0000 7fa6a913b6c0 1 -- 10.20.193.27:0/3737453861 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa6a410f5d0 con 0x7fa6a4115410 2026-02-13T02:25:25.122 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.120+0000 7fa67bfff6c0 1 -- 10.20.193.27:0/3737453861 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7fa68c0a96f0 con 0x7fa6a4115410 2026-02-13T02:25:25.123 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:25:25.123 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:25:25.124 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.121+0000 7fa6a913b6c0 1 -- 10.20.193.27:0/3737453861 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa6740b5600 msgr2=0x7fa6740d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:25.124 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.121+0000 7fa6a913b6c0 1 --2- 10.20.193.27:0/3737453861 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa6740b5600 0x7fa6740d5ab0 secure :-1 s=READY pgs=282 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa698004750 tx=0x7fa698009630 comp rx=0 tx=0).stop 2026-02-13T02:25:25.124 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.121+0000 7fa6a913b6c0 1 -- 10.20.193.27:0/3737453861 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa6a4115410 msgr2=0x7fa6a411d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:25.124 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.121+0000 7fa6a913b6c0 1 --2- 10.20.193.27:0/3737453861 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa6a4115410 0x7fa6a411d7d0 secure :-1 s=READY pgs=437 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa68c005dc0 tx=0x7fa68c00bbb0 comp rx=0 tx=0).stop 2026-02-13T02:25:25.124 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.121+0000 7fa6a913b6c0 1 -- 10.20.193.27:0/3737453861 shutdown_connections 2026-02-13T02:25:25.124 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.121+0000 7fa6a913b6c0 1 --2- 10.20.193.27:0/3737453861 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fa6740b5600 0x7fa6740d5ab0 unknown :-1 s=CLOSED pgs=282 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:25.124 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.121+0000 7fa6a913b6c0 1 --2- 10.20.193.27:0/3737453861 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa6a4122b90 0x7fa6a4123030 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:25.125 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.121+0000 7fa6a913b6c0 1 --2- 10.20.193.27:0/3737453861 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fa6a4115410 0x7fa6a411d7d0 unknown :-1 s=CLOSED pgs=437 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:25.125 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.121+0000 7fa6a913b6c0 1 --2- 10.20.193.27:0/3737453861 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fa6a410f200 0x7fa6a411d290 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:25.125 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.121+0000 7fa6a913b6c0 1 -- 10.20.193.27:0/3737453861 >> 10.20.193.27:0/3737453861 conn(0x7fa6a40ecb00 msgr2=0x7fa6a410cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:25.125 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.121+0000 7fa6a913b6c0 1 -- 10.20.193.27:0/3737453861 shutdown_connections 2026-02-13T02:25:25.125 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:25.121+0000 7fa6a913b6c0 1 -- 10.20.193.27:0/3737453861 wait complete. 2026-02-13T02:25:25.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:25 trial027 ceph-mon[24567]: pgmap v283: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:25:25.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:25 trial028 ceph-mon[26701]: pgmap v283: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:25:25.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:25 trial107 ceph-mon[27422]: pgmap v283: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:25:26.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:26 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3737453861' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:26.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:26 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3737453861' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:26.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:26 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3737453861' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:27.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:27 trial027 ceph-mon[24567]: pgmap v284: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:25:27.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:27 trial028 ceph-mon[26701]: pgmap v284: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:25:27.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:27 trial107 ceph-mon[27422]: pgmap v284: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:25:28.166 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:25:28.288 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:25:28.405 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.400+0000 7f11e4a236c0 1 -- 10.20.193.27:0/2067351345 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f11e0115410 msgr2=0x7f11e0175db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:28.405 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.400+0000 7f11e4a236c0 1 --2- 10.20.193.27:0/2067351345 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f11e0115410 0x7f11e0175db0 secure :-1 s=READY pgs=438 gs=1 cs=0 l=1 c_cookie=16273642303448800595 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f11d000aa70 tx=0x7f11d002fde0 comp rx=0 tx=0).stop 2026-02-13T02:25:28.405 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.400+0000 7f11e4a236c0 1 -- 10.20.193.27:0/2067351345 shutdown_connections 2026-02-13T02:25:28.405 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.400+0000 7f11e4a236c0 1 --2- 10.20.193.27:0/2067351345 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f11e0115410 0x7f11e0175db0 unknown :-1 s=CLOSED pgs=438 gs=1 cs=0 l=1 c_cookie=16273642303448800595 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:28.405 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.400+0000 7f11e4a236c0 1 --2- 10.20.193.27:0/2067351345 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f11e010fbf0 0x7f11e0114c30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:28.405 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.400+0000 7f11e4a236c0 1 --2- 10.20.193.27:0/2067351345 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f11e010f200 0x7f11e010f5d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:28.405 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.400+0000 7f11e4a236c0 1 -- 10.20.193.27:0/2067351345 >> 10.20.193.27:0/2067351345 conn(0x7f11e00ecb00 msgr2=0x7f11e010cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:28.405 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.400+0000 7f11e4a236c0 1 -- 10.20.193.27:0/2067351345 shutdown_connections 2026-02-13T02:25:28.405 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.400+0000 7f11e4a236c0 1 -- 10.20.193.27:0/2067351345 wait complete. 2026-02-13T02:25:28.406 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.401+0000 7f11e4a236c0 1 Processor -- start 2026-02-13T02:25:28.406 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.401+0000 7f11e4a236c0 1 -- start start 2026-02-13T02:25:28.406 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.401+0000 7f11e4a236c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f11e010f200 0x7f11e011d1f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:28.406 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.401+0000 7f11e4a236c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f11e0115410 0x7f11e011d730 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:28.406 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.401+0000 7f11e4a236c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f11e0122af0 0x7f11e0122f90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:28.406 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.401+0000 7f11e4a236c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f11e0123500 con 0x7f11e010f200 2026-02-13T02:25:28.406 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.401+0000 7f11e4a236c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f11e0123670 con 0x7f11e0115410 2026-02-13T02:25:28.406 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.401+0000 7f11e4a236c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f11e0123830 con 0x7f11e0122af0 2026-02-13T02:25:28.406 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.401+0000 7f11de5756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f11e010f200 0x7f11e011d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:28.406 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.401+0000 7f11de5756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f11e010f200 0x7f11e011d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:54448/0 (socket says 10.20.193.27:54448) 2026-02-13T02:25:28.407 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.401+0000 7f11de5756c0 1 -- 10.20.193.27:0/3426156319 learned_addr learned my addr 10.20.193.27:0/3426156319 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:25:28.407 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.401+0000 7f11ddd746c0 1 --2- 10.20.193.27:0/3426156319 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f11e0115410 0x7f11e011d730 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:28.407 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.401+0000 7f11de5756c0 1 -- 10.20.193.27:0/3426156319 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f11e0122af0 msgr2=0x7f11e0122f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:28.407 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.401+0000 7f11de5756c0 1 --2- 10.20.193.27:0/3426156319 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f11e0122af0 0x7f11e0122f90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:28.407 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.401+0000 7f11de5756c0 1 -- 10.20.193.27:0/3426156319 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f11e0115410 msgr2=0x7f11e011d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:28.407 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.401+0000 7f11de5756c0 1 --2- 10.20.193.27:0/3426156319 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f11e0115410 0x7f11e011d730 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:28.407 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.401+0000 7f11de5756c0 1 -- 10.20.193.27:0/3426156319 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d000a660 con 0x7f11e010f200 2026-02-13T02:25:28.407 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.401+0000 7f11de5756c0 1 --2- 10.20.193.27:0/3426156319 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f11e010f200 0x7f11e011d1f0 secure :-1 s=READY pgs=439 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f11d400b710 tx=0x7f11d400bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:28.407 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.401+0000 7f11c77fe6c0 1 -- 10.20.193.27:0/3426156319 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f11d4004250 con 0x7f11e010f200 2026-02-13T02:25:28.407 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.401+0000 7f11e4a236c0 1 -- 10.20.193.27:0/3426156319 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e0163ce0 con 0x7f11e010f200 2026-02-13T02:25:28.407 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.401+0000 7f11c77fe6c0 1 -- 10.20.193.27:0/3426156319 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f11d4002d80 con 0x7f11e010f200 2026-02-13T02:25:28.407 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.401+0000 7f11c77fe6c0 1 -- 10.20.193.27:0/3426156319 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f11d400cdc0 con 0x7f11e010f200 2026-02-13T02:25:28.407 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.401+0000 7f11e4a236c0 1 -- 10.20.193.27:0/3426156319 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e01641c0 con 0x7f11e010f200 2026-02-13T02:25:28.408 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.402+0000 7f11e4a236c0 1 -- 10.20.193.27:0/3426156319 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11e01104d0 con 0x7f11e010f200 2026-02-13T02:25:28.409 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.403+0000 7f11c77fe6c0 1 -- 10.20.193.27:0/3426156319 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f11d40028c0 con 0x7f11e010f200 2026-02-13T02:25:28.409 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.404+0000 7f11c77fe6c0 1 --2- 10.20.193.27:0/3426156319 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f11b80b5600 0x7f11b80d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:28.409 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.404+0000 7f11ddd746c0 1 --2- 10.20.193.27:0/3426156319 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f11b80b5600 0x7f11b80d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:28.409 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.404+0000 7f11c77fe6c0 1 -- 10.20.193.27:0/3426156319 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f11d40e3c30 con 0x7f11e010f200 2026-02-13T02:25:28.409 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.404+0000 7f11c77fe6c0 1 -- 10.20.193.27:0/3426156319 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f11d40a66b0 con 0x7f11e010f200 2026-02-13T02:25:28.409 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.404+0000 7f11ddd746c0 1 --2- 10.20.193.27:0/3426156319 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f11b80b5600 0x7f11b80d5ab0 secure :-1 s=READY pgs=283 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f11e0123b80 tx=0x7f11c8009590 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:28.520 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.518+0000 7f11e4a236c0 1 -- 10.20.193.27:0/3426156319 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f11e010f5d0 con 0x7f11e010f200 2026-02-13T02:25:28.520 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.518+0000 7f11c77fe6c0 1 -- 10.20.193.27:0/3426156319 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f11d40a9860 con 0x7f11e010f200 2026-02-13T02:25:28.521 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:25:28.521 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:25:28.522 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.519+0000 7f11e4a236c0 1 -- 10.20.193.27:0/3426156319 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f11b80b5600 msgr2=0x7f11b80d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:28.522 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.519+0000 7f11e4a236c0 1 --2- 10.20.193.27:0/3426156319 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f11b80b5600 0x7f11b80d5ab0 secure :-1 s=READY pgs=283 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f11e0123b80 tx=0x7f11c8009590 comp rx=0 tx=0).stop 2026-02-13T02:25:28.522 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.519+0000 7f11e4a236c0 1 -- 10.20.193.27:0/3426156319 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f11e010f200 msgr2=0x7f11e011d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:28.522 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.519+0000 7f11e4a236c0 1 --2- 10.20.193.27:0/3426156319 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f11e010f200 0x7f11e011d1f0 secure :-1 s=READY pgs=439 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f11d400b710 tx=0x7f11d400bbe0 comp rx=0 tx=0).stop 2026-02-13T02:25:28.522 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.519+0000 7f11e4a236c0 1 -- 10.20.193.27:0/3426156319 shutdown_connections 2026-02-13T02:25:28.522 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.519+0000 7f11e4a236c0 1 --2- 10.20.193.27:0/3426156319 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f11b80b5600 0x7f11b80d5ab0 unknown :-1 s=CLOSED pgs=283 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:28.522 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.519+0000 7f11e4a236c0 1 --2- 10.20.193.27:0/3426156319 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f11e0122af0 0x7f11e0122f90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:28.523 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.519+0000 7f11e4a236c0 1 --2- 10.20.193.27:0/3426156319 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f11e0115410 0x7f11e011d730 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:28.523 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.519+0000 7f11e4a236c0 1 --2- 10.20.193.27:0/3426156319 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f11e010f200 0x7f11e011d1f0 unknown :-1 s=CLOSED pgs=439 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:28.523 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.519+0000 7f11e4a236c0 1 -- 10.20.193.27:0/3426156319 >> 10.20.193.27:0/3426156319 conn(0x7f11e00ecb00 msgr2=0x7f11e010cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:28.523 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.519+0000 7f11e4a236c0 1 -- 10.20.193.27:0/3426156319 shutdown_connections 2026-02-13T02:25:28.523 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:28.519+0000 7f11e4a236c0 1 -- 10.20.193.27:0/3426156319 wait complete. 2026-02-13T02:25:29.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:29 trial027 ceph-mon[24567]: pgmap v285: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:25:29.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:29 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3426156319' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:29.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:29 trial028 ceph-mon[26701]: pgmap v285: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:25:29.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:29 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3426156319' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:29.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:29 trial107 ceph-mon[27422]: pgmap v285: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:25:29.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:29 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3426156319' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:31.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:31 trial027 ceph-mon[24567]: pgmap v286: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 964 B/s rd, 1 op/s 2026-02-13T02:25:31.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:31 trial028 ceph-mon[26701]: pgmap v286: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 964 B/s rd, 1 op/s 2026-02-13T02:25:31.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:31 trial107 ceph-mon[27422]: pgmap v286: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 964 B/s rd, 1 op/s 2026-02-13T02:25:31.562 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:25:31.683 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:25:31.796 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.792+0000 7f832221d6c0 1 -- 10.20.193.27:0/3444827485 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f831c115440 msgr2=0x7f831c175de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:31.797 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.792+0000 7f832221d6c0 1 --2- 10.20.193.27:0/3444827485 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f831c115440 0x7f831c175de0 secure :-1 s=READY pgs=440 gs=1 cs=0 l=1 c_cookie=2982033747254031180 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f831000aa70 tx=0x7f831002fde0 comp rx=0 tx=0).stop 2026-02-13T02:25:31.797 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.792+0000 7f832221d6c0 1 -- 10.20.193.27:0/3444827485 shutdown_connections 2026-02-13T02:25:31.797 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.792+0000 7f832221d6c0 1 --2- 10.20.193.27:0/3444827485 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f831c115440 0x7f831c175de0 unknown :-1 s=CLOSED pgs=440 gs=1 cs=0 l=1 c_cookie=2982033747254031180 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:31.797 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.792+0000 7f832221d6c0 1 --2- 10.20.193.27:0/3444827485 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f831c10fc20 0x7f831c114c60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:31.797 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.792+0000 7f832221d6c0 1 --2- 10.20.193.27:0/3444827485 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f831c10f230 0x7f831c10f600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:31.797 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.792+0000 7f832221d6c0 1 -- 10.20.193.27:0/3444827485 >> 10.20.193.27:0/3444827485 conn(0x7f831c0ecb30 msgr2=0x7f831c10cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:31.797 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.792+0000 7f832221d6c0 1 -- 10.20.193.27:0/3444827485 shutdown_connections 2026-02-13T02:25:31.797 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.792+0000 7f832221d6c0 1 -- 10.20.193.27:0/3444827485 wait complete. 2026-02-13T02:25:31.797 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.792+0000 7f832221d6c0 1 Processor -- start 2026-02-13T02:25:31.798 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.792+0000 7f832221d6c0 1 -- start start 2026-02-13T02:25:31.798 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.792+0000 7f832221d6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f831c10f230 0x7f831c11d200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:31.798 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.792+0000 7f832221d6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f831c115440 0x7f831c11d740 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:31.798 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.792+0000 7f832221d6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f831c122b00 0x7f831c122fa0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:31.798 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.792+0000 7f832221d6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f831c123510 con 0x7f831c115440 2026-02-13T02:25:31.798 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.792+0000 7f832221d6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f831c123680 con 0x7f831c10f230 2026-02-13T02:25:31.798 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.792+0000 7f832221d6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f831c123840 con 0x7f831c122b00 2026-02-13T02:25:31.798 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.792+0000 7f831affd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f831c115440 0x7f831c11d740 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:31.798 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.792+0000 7f831affd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f831c115440 0x7f831c11d740 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:54466/0 (socket says 10.20.193.27:54466) 2026-02-13T02:25:31.798 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.792+0000 7f831affd6c0 1 -- 10.20.193.27:0/3575320655 learned_addr learned my addr 10.20.193.27:0/3575320655 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:25:31.798 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.793+0000 7f831b7fe6c0 1 --2- 10.20.193.27:0/3575320655 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f831c10f230 0x7f831c11d200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:31.799 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.793+0000 7f831bfff6c0 1 --2- 10.20.193.27:0/3575320655 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f831c122b00 0x7f831c122fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:31.799 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.793+0000 7f831affd6c0 1 -- 10.20.193.27:0/3575320655 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f831c122b00 msgr2=0x7f831c122fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:31.799 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.793+0000 7f831affd6c0 1 --2- 10.20.193.27:0/3575320655 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f831c122b00 0x7f831c122fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:31.799 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.793+0000 7f831affd6c0 1 -- 10.20.193.27:0/3575320655 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f831c10f230 msgr2=0x7f831c11d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:31.799 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.793+0000 7f831affd6c0 1 --2- 10.20.193.27:0/3575320655 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f831c10f230 0x7f831c11d200 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:31.799 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.793+0000 7f831affd6c0 1 -- 10.20.193.27:0/3575320655 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831000a660 con 0x7f831c115440 2026-02-13T02:25:31.799 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.793+0000 7f831bfff6c0 1 --2- 10.20.193.27:0/3575320655 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f831c122b00 0x7f831c122fa0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:25:31.799 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.793+0000 7f831affd6c0 1 --2- 10.20.193.27:0/3575320655 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f831c115440 0x7f831c11d740 secure :-1 s=READY pgs=441 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f830c005dc0 tx=0x7f830c00bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:31.799 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.793+0000 7f8318ff96c0 1 -- 10.20.193.27:0/3575320655 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f830c004250 con 0x7f831c115440 2026-02-13T02:25:31.799 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.793+0000 7f832221d6c0 1 -- 10.20.193.27:0/3575320655 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f831c163cf0 con 0x7f831c115440 2026-02-13T02:25:31.799 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.793+0000 7f8318ff96c0 1 -- 10.20.193.27:0/3575320655 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f830c009070 con 0x7f831c115440 2026-02-13T02:25:31.799 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.793+0000 7f8318ff96c0 1 -- 10.20.193.27:0/3575320655 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f830c00cd50 con 0x7f831c115440 2026-02-13T02:25:31.800 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.793+0000 7f832221d6c0 1 -- 10.20.193.27:0/3575320655 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f831c164180 con 0x7f831c115440 2026-02-13T02:25:31.800 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.793+0000 7f832221d6c0 1 -- 10.20.193.27:0/3575320655 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f831c110500 con 0x7f831c115440 2026-02-13T02:25:31.801 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.795+0000 7f8318ff96c0 1 -- 10.20.193.27:0/3575320655 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f830c0028c0 con 0x7f831c115440 2026-02-13T02:25:31.801 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.796+0000 7f8318ff96c0 1 --2- 10.20.193.27:0/3575320655 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f82f00b5600 0x7f82f00d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:31.801 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.796+0000 7f831b7fe6c0 1 --2- 10.20.193.27:0/3575320655 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f82f00b5600 0x7f82f00d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:31.801 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.796+0000 7f8318ff96c0 1 -- 10.20.193.27:0/3575320655 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f830c0e3b20 con 0x7f831c115440 2026-02-13T02:25:31.801 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.796+0000 7f8318ff96c0 1 -- 10.20.193.27:0/3575320655 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f830c0aa000 con 0x7f831c115440 2026-02-13T02:25:31.801 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.796+0000 7f831b7fe6c0 1 --2- 10.20.193.27:0/3575320655 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f82f00b5600 0x7f82f00d5ab0 secure :-1 s=READY pgs=284 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8308005980 tx=0x7f8308005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:31.913 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.911+0000 7f832221d6c0 1 -- 10.20.193.27:0/3575320655 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f831c10fc20 con 0x7f831c115440 2026-02-13T02:25:31.913 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.911+0000 7f8318ff96c0 1 -- 10.20.193.27:0/3575320655 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f830c0ac070 con 0x7f831c115440 2026-02-13T02:25:31.914 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:25:31.914 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:25:31.915 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.912+0000 7f832221d6c0 1 -- 10.20.193.27:0/3575320655 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f82f00b5600 msgr2=0x7f82f00d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:31.915 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.912+0000 7f832221d6c0 1 --2- 10.20.193.27:0/3575320655 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f82f00b5600 0x7f82f00d5ab0 secure :-1 s=READY pgs=284 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8308005980 tx=0x7f8308005910 comp rx=0 tx=0).stop 2026-02-13T02:25:31.915 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.912+0000 7f832221d6c0 1 -- 10.20.193.27:0/3575320655 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f831c115440 msgr2=0x7f831c11d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:31.915 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.912+0000 7f832221d6c0 1 --2- 10.20.193.27:0/3575320655 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f831c115440 0x7f831c11d740 secure :-1 s=READY pgs=441 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f830c005dc0 tx=0x7f830c00bbb0 comp rx=0 tx=0).stop 2026-02-13T02:25:31.915 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.912+0000 7f832221d6c0 1 -- 10.20.193.27:0/3575320655 shutdown_connections 2026-02-13T02:25:31.915 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.912+0000 7f832221d6c0 1 --2- 10.20.193.27:0/3575320655 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f82f00b5600 0x7f82f00d5ab0 unknown :-1 s=CLOSED pgs=284 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:31.915 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.912+0000 7f832221d6c0 1 --2- 10.20.193.27:0/3575320655 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f831c122b00 0x7f831c122fa0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:31.915 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.912+0000 7f832221d6c0 1 --2- 10.20.193.27:0/3575320655 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f831c115440 0x7f831c11d740 unknown :-1 s=CLOSED pgs=441 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:31.915 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.912+0000 7f832221d6c0 1 --2- 10.20.193.27:0/3575320655 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f831c10f230 0x7f831c11d200 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:31.916 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.912+0000 7f832221d6c0 1 -- 10.20.193.27:0/3575320655 >> 10.20.193.27:0/3575320655 conn(0x7f831c0ecb30 msgr2=0x7f831c10cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:31.916 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.912+0000 7f832221d6c0 1 -- 10.20.193.27:0/3575320655 shutdown_connections 2026-02-13T02:25:31.916 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:31.912+0000 7f832221d6c0 1 -- 10.20.193.27:0/3575320655 wait complete. 2026-02-13T02:25:32.394 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:32 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3575320655' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:32.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:32 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3575320655' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:32.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:32 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3575320655' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:33.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:33 trial027 ceph-mon[24567]: pgmap v287: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:33.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:33 trial028 ceph-mon[26701]: pgmap v287: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:33.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:33 trial107 ceph-mon[27422]: pgmap v287: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:34.956 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:25:35.075 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:25:35.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.188+0000 7f49b53a76c0 1 -- 10.20.193.27:0/2482127938 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f49b010f200 msgr2=0x7f49b010f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:35.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.188+0000 7f49b53a76c0 1 --2- 10.20.193.27:0/2482127938 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f49b010f200 0x7f49b010f5d0 secure :-1 s=READY pgs=442 gs=1 cs=0 l=1 c_cookie=8732573873684450374 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f49a40099a0 tx=0x7f49a402ed20 comp rx=0 tx=0).stop 2026-02-13T02:25:35.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.188+0000 7f49b53a76c0 1 -- 10.20.193.27:0/2482127938 shutdown_connections 2026-02-13T02:25:35.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.188+0000 7f49b53a76c0 1 --2- 10.20.193.27:0/2482127938 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f49b0115410 0x7f49b0175db0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:35.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.188+0000 7f49b53a76c0 1 --2- 10.20.193.27:0/2482127938 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f49b010fbf0 0x7f49b0114c30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:35.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.188+0000 7f49b53a76c0 1 --2- 10.20.193.27:0/2482127938 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f49b010f200 0x7f49b010f5d0 unknown :-1 s=CLOSED pgs=442 gs=1 cs=0 l=1 c_cookie=8732573873684450374 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:35.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.188+0000 7f49b53a76c0 1 -- 10.20.193.27:0/2482127938 >> 10.20.193.27:0/2482127938 conn(0x7f49b00ecb00 msgr2=0x7f49b010cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:35.192 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.188+0000 7f49b53a76c0 1 -- 10.20.193.27:0/2482127938 shutdown_connections 2026-02-13T02:25:35.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.188+0000 7f49b53a76c0 1 -- 10.20.193.27:0/2482127938 wait complete. 2026-02-13T02:25:35.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.189+0000 7f49b53a76c0 1 Processor -- start 2026-02-13T02:25:35.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.189+0000 7f49b53a76c0 1 -- start start 2026-02-13T02:25:35.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.189+0000 7f49b53a76c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f49b010f200 0x7f49b011d310 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:35.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.189+0000 7f49b53a76c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f49b0115410 0x7f49b011d850 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:35.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.189+0000 7f49b53a76c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f49b0122c10 0x7f49b01230b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:35.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.189+0000 7f49b53a76c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f49b0123620 con 0x7f49b010f200 2026-02-13T02:25:35.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.189+0000 7f49b53a76c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f49b0123790 con 0x7f49b0122c10 2026-02-13T02:25:35.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.189+0000 7f49b53a76c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f49b0123950 con 0x7f49b0115410 2026-02-13T02:25:35.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.189+0000 7f49aeffd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f49b010f200 0x7f49b011d310 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:35.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.189+0000 7f49aeffd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f49b010f200 0x7f49b011d310 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:54496/0 (socket says 10.20.193.27:54496) 2026-02-13T02:25:35.193 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.189+0000 7f49aeffd6c0 1 -- 10.20.193.27:0/3840026169 learned_addr learned my addr 10.20.193.27:0/3840026169 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:25:35.194 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.189+0000 7f49af7fe6c0 1 --2- 10.20.193.27:0/3840026169 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f49b0122c10 0x7f49b01230b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:35.194 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.189+0000 7f49aeffd6c0 1 -- 10.20.193.27:0/3840026169 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f49b0115410 msgr2=0x7f49b011d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:35.194 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.189+0000 7f49aeffd6c0 1 --2- 10.20.193.27:0/3840026169 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f49b0115410 0x7f49b011d850 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:35.194 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.189+0000 7f49aeffd6c0 1 -- 10.20.193.27:0/3840026169 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f49b0122c10 msgr2=0x7f49b01230b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:35.194 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.189+0000 7f49aeffd6c0 1 --2- 10.20.193.27:0/3840026169 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f49b0122c10 0x7f49b01230b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:35.194 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.189+0000 7f49aeffd6c0 1 -- 10.20.193.27:0/3840026169 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a4009590 con 0x7f49b010f200 2026-02-13T02:25:35.194 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.189+0000 7f49aeffd6c0 1 --2- 10.20.193.27:0/3840026169 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f49b010f200 0x7f49b011d310 secure :-1 s=READY pgs=443 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f49a402f230 tx=0x7f49a402fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:35.194 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.189+0000 7f498ffff6c0 1 -- 10.20.193.27:0/3840026169 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f49a4002ba0 con 0x7f49b010f200 2026-02-13T02:25:35.194 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.189+0000 7f498ffff6c0 1 -- 10.20.193.27:0/3840026169 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f49a4002d00 con 0x7f49b010f200 2026-02-13T02:25:35.194 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.189+0000 7f49b53a76c0 1 -- 10.20.193.27:0/3840026169 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49b0163da0 con 0x7f49b010f200 2026-02-13T02:25:35.194 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.189+0000 7f498ffff6c0 1 -- 10.20.193.27:0/3840026169 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f49a40372f0 con 0x7f49b010f200 2026-02-13T02:25:35.194 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.189+0000 7f49b53a76c0 1 -- 10.20.193.27:0/3840026169 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49b01641d0 con 0x7f49b010f200 2026-02-13T02:25:35.194 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.190+0000 7f49b53a76c0 1 -- 10.20.193.27:0/3840026169 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49b01104d0 con 0x7f49b010f200 2026-02-13T02:25:35.195 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.191+0000 7f498ffff6c0 1 -- 10.20.193.27:0/3840026169 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f49a40078d0 con 0x7f49b010f200 2026-02-13T02:25:35.196 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.192+0000 7f498ffff6c0 1 --2- 10.20.193.27:0/3840026169 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f49840b5600 0x7f49840d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:35.196 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.192+0000 7f498ffff6c0 1 -- 10.20.193.27:0/3840026169 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f49a4107c30 con 0x7f49b010f200 2026-02-13T02:25:35.196 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.192+0000 7f49ae7fc6c0 1 --2- 10.20.193.27:0/3840026169 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f49840b5600 0x7f49840d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:35.196 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.192+0000 7f498ffff6c0 1 -- 10.20.193.27:0/3840026169 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f49a40cd680 con 0x7f49b010f200 2026-02-13T02:25:35.196 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.192+0000 7f49ae7fc6c0 1 --2- 10.20.193.27:0/3840026169 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f49840b5600 0x7f49840d5ab0 secure :-1 s=READY pgs=285 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4998005980 tx=0x7f4998005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:35.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.306+0000 7f49b53a76c0 1 -- 10.20.193.27:0/3840026169 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f49b010f5d0 con 0x7f49b010f200 2026-02-13T02:25:35.308 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:35 trial027 ceph-mon[24567]: pgmap v288: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:35.309 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.307+0000 7f498ffff6c0 1 -- 10.20.193.27:0/3840026169 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f49a40cd860 con 0x7f49b010f200 2026-02-13T02:25:35.309 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:25:35.309 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:25:35.310 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.308+0000 7f49b53a76c0 1 -- 10.20.193.27:0/3840026169 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f49840b5600 msgr2=0x7f49840d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:35.310 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.308+0000 7f49b53a76c0 1 --2- 10.20.193.27:0/3840026169 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f49840b5600 0x7f49840d5ab0 secure :-1 s=READY pgs=285 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4998005980 tx=0x7f4998005910 comp rx=0 tx=0).stop 2026-02-13T02:25:35.310 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.308+0000 7f49b53a76c0 1 -- 10.20.193.27:0/3840026169 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f49b010f200 msgr2=0x7f49b011d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:35.310 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.308+0000 7f49b53a76c0 1 --2- 10.20.193.27:0/3840026169 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f49b010f200 0x7f49b011d310 secure :-1 s=READY pgs=443 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f49a402f230 tx=0x7f49a402fd40 comp rx=0 tx=0).stop 2026-02-13T02:25:35.311 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.308+0000 7f49b53a76c0 1 -- 10.20.193.27:0/3840026169 shutdown_connections 2026-02-13T02:25:35.311 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.308+0000 7f49b53a76c0 1 --2- 10.20.193.27:0/3840026169 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f49840b5600 0x7f49840d5ab0 unknown :-1 s=CLOSED pgs=285 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:35.311 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.308+0000 7f49b53a76c0 1 --2- 10.20.193.27:0/3840026169 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f49b0122c10 0x7f49b01230b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:35.311 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.308+0000 7f49b53a76c0 1 --2- 10.20.193.27:0/3840026169 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f49b0115410 0x7f49b011d850 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:35.311 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.308+0000 7f49b53a76c0 1 --2- 10.20.193.27:0/3840026169 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f49b010f200 0x7f49b011d310 unknown :-1 s=CLOSED pgs=443 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:35.311 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.308+0000 7f49b53a76c0 1 -- 10.20.193.27:0/3840026169 >> 10.20.193.27:0/3840026169 conn(0x7f49b00ecb00 msgr2=0x7f49b010cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:35.311 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.308+0000 7f49b53a76c0 1 -- 10.20.193.27:0/3840026169 shutdown_connections 2026-02-13T02:25:35.311 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:35.308+0000 7f49b53a76c0 1 -- 10.20.193.27:0/3840026169 wait complete. 2026-02-13T02:25:35.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:35 trial028 ceph-mon[26701]: pgmap v288: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:35.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:35 trial107 ceph-mon[27422]: pgmap v288: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:36.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:36 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3840026169' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:36.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:36 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3840026169' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:36.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:36 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3840026169' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:37.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:37 trial027 ceph-mon[24567]: pgmap v289: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:37.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:37 trial028 ceph-mon[26701]: pgmap v289: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:37.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:37 trial107 ceph-mon[27422]: pgmap v289: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:38.351 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:25:38.473 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:25:38.593 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.590+0000 7f184fe576c0 1 -- 10.20.193.27:0/2012695614 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f184810fbf0 msgr2=0x7f1848114c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:38.594 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.590+0000 7f184fe576c0 1 --2- 10.20.193.27:0/2012695614 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f184810fbf0 0x7f1848114c30 secure :-1 s=READY pgs=444 gs=1 cs=0 l=1 c_cookie=859157635985850353 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f183c0099a0 tx=0x7f183c02ed20 comp rx=0 tx=0).stop 2026-02-13T02:25:38.594 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.590+0000 7f184fe576c0 1 -- 10.20.193.27:0/2012695614 shutdown_connections 2026-02-13T02:25:38.594 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.590+0000 7f184fe576c0 1 --2- 10.20.193.27:0/2012695614 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1848115410 0x7f1848175db0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:38.594 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.590+0000 7f184fe576c0 1 --2- 10.20.193.27:0/2012695614 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f184810fbf0 0x7f1848114c30 unknown :-1 s=CLOSED pgs=444 gs=1 cs=0 l=1 c_cookie=859157635985850353 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:38.594 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.590+0000 7f184fe576c0 1 --2- 10.20.193.27:0/2012695614 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f184810f200 0x7f184810f5d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:38.594 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.590+0000 7f184fe576c0 1 -- 10.20.193.27:0/2012695614 >> 10.20.193.27:0/2012695614 conn(0x7f18480ecb00 msgr2=0x7f184810cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:38.594 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.590+0000 7f184fe576c0 1 -- 10.20.193.27:0/2012695614 shutdown_connections 2026-02-13T02:25:38.594 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.590+0000 7f184fe576c0 1 -- 10.20.193.27:0/2012695614 wait complete. 2026-02-13T02:25:38.594 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.591+0000 7f184fe576c0 1 Processor -- start 2026-02-13T02:25:38.595 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.591+0000 7f184fe576c0 1 -- start start 2026-02-13T02:25:38.595 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.591+0000 7f184fe576c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f184810f200 0x7f18480f4ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:38.595 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.591+0000 7f184fe576c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1848115410 0x7f1848175cb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:38.595 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.591+0000 7f184fe576c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f18480f5310 0x7f18480fa5e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:38.595 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.591+0000 7f184fe576c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f18480f5a00 con 0x7f18480f5310 2026-02-13T02:25:38.595 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.591+0000 7f184fe576c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f18480fab20 con 0x7f184810f200 2026-02-13T02:25:38.595 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.591+0000 7f184fe576c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f18480face0 con 0x7f1848115410 2026-02-13T02:25:38.595 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.591+0000 7f184e3c96c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f18480f5310 0x7f18480fa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:38.595 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.591+0000 7f184e3c96c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f18480f5310 0x7f18480fa5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:58096/0 (socket says 10.20.193.27:58096) 2026-02-13T02:25:38.595 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.591+0000 7f184e3c96c0 1 -- 10.20.193.27:0/3404495350 learned_addr learned my addr 10.20.193.27:0/3404495350 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:25:38.595 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.591+0000 7f184dbc86c0 1 --2- 10.20.193.27:0/3404495350 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f184810f200 0x7f18480f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:38.595 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.591+0000 7f184d3c76c0 1 --2- 10.20.193.27:0/3404495350 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1848115410 0x7f1848175cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:38.596 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.591+0000 7f184e3c96c0 1 -- 10.20.193.27:0/3404495350 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1848115410 msgr2=0x7f1848175cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:38.596 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.591+0000 7f184e3c96c0 1 --2- 10.20.193.27:0/3404495350 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1848115410 0x7f1848175cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:38.596 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.591+0000 7f184e3c96c0 1 -- 10.20.193.27:0/3404495350 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f184810f200 msgr2=0x7f18480f4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:38.596 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.591+0000 7f184e3c96c0 1 --2- 10.20.193.27:0/3404495350 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f184810f200 0x7f18480f4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:38.596 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.591+0000 7f184e3c96c0 1 -- 10.20.193.27:0/3404495350 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183c009590 con 0x7f18480f5310 2026-02-13T02:25:38.596 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.591+0000 7f184e3c96c0 1 --2- 10.20.193.27:0/3404495350 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f18480f5310 0x7f18480fa5e0 secure :-1 s=READY pgs=445 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1844006ea0 tx=0x7f184400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:38.596 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.591+0000 7f1836ffd6c0 1 -- 10.20.193.27:0/3404495350 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1844003a30 con 0x7f18480f5310 2026-02-13T02:25:38.596 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.591+0000 7f184fe576c0 1 -- 10.20.193.27:0/3404495350 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18480fafc0 con 0x7f18480f5310 2026-02-13T02:25:38.596 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.591+0000 7f1836ffd6c0 1 -- 10.20.193.27:0/3404495350 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f18440054f0 con 0x7f18480f5310 2026-02-13T02:25:38.596 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.591+0000 7f1836ffd6c0 1 -- 10.20.193.27:0/3404495350 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f184400dcd0 con 0x7f18480f5310 2026-02-13T02:25:38.596 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.591+0000 7f184fe576c0 1 -- 10.20.193.27:0/3404495350 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18480fb750 con 0x7f18480f5310 2026-02-13T02:25:38.596 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.592+0000 7f184fe576c0 1 -- 10.20.193.27:0/3404495350 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18481104d0 con 0x7f18480f5310 2026-02-13T02:25:38.597 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.593+0000 7f1836ffd6c0 1 -- 10.20.193.27:0/3404495350 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f1844003b90 con 0x7f18480f5310 2026-02-13T02:25:38.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.594+0000 7f1836ffd6c0 1 --2- 10.20.193.27:0/3404495350 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f18240b5600 0x7f18240d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:38.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.594+0000 7f1836ffd6c0 1 -- 10.20.193.27:0/3404495350 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f18440e4a20 con 0x7f18480f5310 2026-02-13T02:25:38.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.594+0000 7f184dbc86c0 1 --2- 10.20.193.27:0/3404495350 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f18240b5600 0x7f18240d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:38.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.594+0000 7f1836ffd6c0 1 -- 10.20.193.27:0/3404495350 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f18440a74e0 con 0x7f18480f5310 2026-02-13T02:25:38.598 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.594+0000 7f184dbc86c0 1 --2- 10.20.193.27:0/3404495350 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f18240b5600 0x7f18240d5ab0 secure :-1 s=READY pgs=286 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1838004750 tx=0x7f1838009390 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:38.710 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:25:38.710 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:25:38.711 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.708+0000 7f184fe576c0 1 -- 10.20.193.27:0/3404495350 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f18480f5e30 con 0x7f18480f5310 2026-02-13T02:25:38.711 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.708+0000 7f1836ffd6c0 1 -- 10.20.193.27:0/3404495350 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f184400de60 con 0x7f18480f5310 2026-02-13T02:25:38.711 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.709+0000 7f184fe576c0 1 -- 10.20.193.27:0/3404495350 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f18240b5600 msgr2=0x7f18240d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:38.711 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.709+0000 7f184fe576c0 1 --2- 10.20.193.27:0/3404495350 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f18240b5600 0x7f18240d5ab0 secure :-1 s=READY pgs=286 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1838004750 tx=0x7f1838009390 comp rx=0 tx=0).stop 2026-02-13T02:25:38.711 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.709+0000 7f184fe576c0 1 -- 10.20.193.27:0/3404495350 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f18480f5310 msgr2=0x7f18480fa5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:38.711 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.709+0000 7f184fe576c0 1 --2- 10.20.193.27:0/3404495350 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f18480f5310 0x7f18480fa5e0 secure :-1 s=READY pgs=445 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1844006ea0 tx=0x7f184400cca0 comp rx=0 tx=0).stop 2026-02-13T02:25:38.711 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.709+0000 7f184fe576c0 1 -- 10.20.193.27:0/3404495350 shutdown_connections 2026-02-13T02:25:38.712 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.709+0000 7f184fe576c0 1 --2- 10.20.193.27:0/3404495350 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f18240b5600 0x7f18240d5ab0 unknown :-1 s=CLOSED pgs=286 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:38.712 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.709+0000 7f184fe576c0 1 --2- 10.20.193.27:0/3404495350 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f18480f5310 0x7f18480fa5e0 unknown :-1 s=CLOSED pgs=445 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:38.712 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.709+0000 7f184fe576c0 1 --2- 10.20.193.27:0/3404495350 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1848115410 0x7f1848175cb0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:38.712 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.709+0000 7f184fe576c0 1 --2- 10.20.193.27:0/3404495350 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f184810f200 0x7f18480f4ce0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:38.712 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.709+0000 7f184fe576c0 1 -- 10.20.193.27:0/3404495350 >> 10.20.193.27:0/3404495350 conn(0x7f18480ecb00 msgr2=0x7f184810d900 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:38.712 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.709+0000 7f184fe576c0 1 -- 10.20.193.27:0/3404495350 shutdown_connections 2026-02-13T02:25:38.712 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:38.709+0000 7f184fe576c0 1 -- 10.20.193.27:0/3404495350 wait complete. 2026-02-13T02:25:39.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:39 trial027 ceph-mon[24567]: pgmap v290: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:39.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:39 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3404495350' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:39.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:39 trial028 ceph-mon[26701]: pgmap v290: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:39.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:39 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3404495350' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:39.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:39 trial107 ceph-mon[27422]: pgmap v290: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:39.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:39 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3404495350' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:41.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:41 trial027 ceph-mon[24567]: pgmap v291: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:41.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:41 trial028 ceph-mon[26701]: pgmap v291: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:41.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:41 trial107 ceph-mon[27422]: pgmap v291: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:41.752 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:25:41.869 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:25:41.986 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.983+0000 7fe759b786c0 1 -- 10.20.193.27:0/2583531853 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe754115440 msgr2=0x7fe754175de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:41.986 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.983+0000 7fe759b786c0 1 --2- 10.20.193.27:0/2583531853 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe754115440 0x7fe754175de0 secure :-1 s=READY pgs=446 gs=1 cs=0 l=1 c_cookie=14609150659974140431 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe74800aa70 tx=0x7fe74802fde0 comp rx=0 tx=0).stop 2026-02-13T02:25:41.986 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.983+0000 7fe759b786c0 1 -- 10.20.193.27:0/2583531853 shutdown_connections 2026-02-13T02:25:41.987 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.983+0000 7fe759b786c0 1 --2- 10.20.193.27:0/2583531853 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe754115440 0x7fe754175de0 unknown :-1 s=CLOSED pgs=446 gs=1 cs=0 l=1 c_cookie=14609150659974140431 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:41.987 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.983+0000 7fe759b786c0 1 --2- 10.20.193.27:0/2583531853 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe75410fc20 0x7fe754114c60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:41.987 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.983+0000 7fe759b786c0 1 --2- 10.20.193.27:0/2583531853 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe75410f230 0x7fe75410f600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:41.987 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.983+0000 7fe759b786c0 1 -- 10.20.193.27:0/2583531853 >> 10.20.193.27:0/2583531853 conn(0x7fe7540ecb30 msgr2=0x7fe75410cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:41.987 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.983+0000 7fe759b786c0 1 -- 10.20.193.27:0/2583531853 shutdown_connections 2026-02-13T02:25:41.987 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.983+0000 7fe759b786c0 1 -- 10.20.193.27:0/2583531853 wait complete. 2026-02-13T02:25:41.987 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.983+0000 7fe759b786c0 1 Processor -- start 2026-02-13T02:25:41.987 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.983+0000 7fe759b786c0 1 -- start start 2026-02-13T02:25:41.987 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.983+0000 7fe759b786c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe75410f230 0x7fe75411d200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:41.987 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.983+0000 7fe759b786c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe754115440 0x7fe75411d740 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:41.987 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.983+0000 7fe759b786c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe754122b00 0x7fe754122fa0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:41.988 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.983+0000 7fe759b786c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe754123510 con 0x7fe754115440 2026-02-13T02:25:41.988 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.983+0000 7fe759b786c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe754123680 con 0x7fe754122b00 2026-02-13T02:25:41.988 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.983+0000 7fe759b786c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe754123840 con 0x7fe75410f230 2026-02-13T02:25:41.988 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.984+0000 7fe752ffd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe754115440 0x7fe75411d740 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:41.988 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.984+0000 7fe752ffd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe754115440 0x7fe75411d740 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:58108/0 (socket says 10.20.193.27:58108) 2026-02-13T02:25:41.988 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.984+0000 7fe752ffd6c0 1 -- 10.20.193.27:0/50719270 learned_addr learned my addr 10.20.193.27:0/50719270 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:25:41.988 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.984+0000 7fe7537fe6c0 1 --2- 10.20.193.27:0/50719270 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe75410f230 0x7fe75411d200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:41.988 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.984+0000 7fe752ffd6c0 1 -- 10.20.193.27:0/50719270 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe75410f230 msgr2=0x7fe75411d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:41.988 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.984+0000 7fe752ffd6c0 1 --2- 10.20.193.27:0/50719270 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe75410f230 0x7fe75411d200 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:41.988 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.984+0000 7fe752ffd6c0 1 -- 10.20.193.27:0/50719270 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe754122b00 msgr2=0x7fe754122fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:41.988 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.984+0000 7fe752ffd6c0 1 --2- 10.20.193.27:0/50719270 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe754122b00 0x7fe754122fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:41.988 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.984+0000 7fe752ffd6c0 1 -- 10.20.193.27:0/50719270 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe74800a660 con 0x7fe754115440 2026-02-13T02:25:41.989 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.984+0000 7fe752ffd6c0 1 --2- 10.20.193.27:0/50719270 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe754115440 0x7fe75411d740 secure :-1 s=READY pgs=447 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe74400b800 tx=0x7fe74400bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:41.989 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.984+0000 7fe750ff96c0 1 -- 10.20.193.27:0/50719270 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe744004a80 con 0x7fe754115440 2026-02-13T02:25:41.989 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.984+0000 7fe750ff96c0 1 -- 10.20.193.27:0/50719270 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fe744009070 con 0x7fe754115440 2026-02-13T02:25:41.989 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.984+0000 7fe759b786c0 1 -- 10.20.193.27:0/50719270 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe754163cf0 con 0x7fe754115440 2026-02-13T02:25:41.989 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.984+0000 7fe750ff96c0 1 -- 10.20.193.27:0/50719270 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe74400cdd0 con 0x7fe754115440 2026-02-13T02:25:41.989 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.984+0000 7fe759b786c0 1 -- 10.20.193.27:0/50719270 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7541641d0 con 0x7fe754115440 2026-02-13T02:25:41.989 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.984+0000 7fe759b786c0 1 -- 10.20.193.27:0/50719270 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe754110500 con 0x7fe754115440 2026-02-13T02:25:41.990 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.985+0000 7fe750ff96c0 1 -- 10.20.193.27:0/50719270 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7fe744004060 con 0x7fe754115440 2026-02-13T02:25:41.990 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.986+0000 7fe750ff96c0 1 --2- 10.20.193.27:0/50719270 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe7280b5680 0x7fe7280d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:41.990 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.986+0000 7fe750ff96c0 1 -- 10.20.193.27:0/50719270 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7fe7440e3a60 con 0x7fe754115440 2026-02-13T02:25:41.990 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.986+0000 7fe7537fe6c0 1 --2- 10.20.193.27:0/50719270 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe7280b5680 0x7fe7280d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:41.990 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.986+0000 7fe7537fe6c0 1 --2- 10.20.193.27:0/50719270 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe7280b5680 0x7fe7280d5b30 secure :-1 s=READY pgs=287 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe740005980 tx=0x7fe740005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:41.991 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:41.987+0000 7fe750ff96c0 1 -- 10.20.193.27:0/50719270 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fe7440a64e0 con 0x7fe754115440 2026-02-13T02:25:42.104 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:42.102+0000 7fe759b786c0 1 -- 10.20.193.27:0/50719270 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe75410f600 con 0x7fe754115440 2026-02-13T02:25:42.104 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:42.102+0000 7fe750ff96c0 1 -- 10.20.193.27:0/50719270 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7fe7440a9690 con 0x7fe754115440 2026-02-13T02:25:42.104 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:25:42.104 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:25:42.106 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:42.104+0000 7fe759b786c0 1 -- 10.20.193.27:0/50719270 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe7280b5680 msgr2=0x7fe7280d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:42.106 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:42.104+0000 7fe759b786c0 1 --2- 10.20.193.27:0/50719270 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe7280b5680 0x7fe7280d5b30 secure :-1 s=READY pgs=287 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe740005980 tx=0x7fe740005910 comp rx=0 tx=0).stop 2026-02-13T02:25:42.106 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:42.104+0000 7fe759b786c0 1 -- 10.20.193.27:0/50719270 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe754115440 msgr2=0x7fe75411d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:42.106 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:42.104+0000 7fe759b786c0 1 --2- 10.20.193.27:0/50719270 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe754115440 0x7fe75411d740 secure :-1 s=READY pgs=447 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe74400b800 tx=0x7fe74400bcd0 comp rx=0 tx=0).stop 2026-02-13T02:25:42.106 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:42.104+0000 7fe759b786c0 1 -- 10.20.193.27:0/50719270 shutdown_connections 2026-02-13T02:25:42.106 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:42.104+0000 7fe759b786c0 1 --2- 10.20.193.27:0/50719270 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fe7280b5680 0x7fe7280d5b30 unknown :-1 s=CLOSED pgs=287 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:42.106 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:42.104+0000 7fe759b786c0 1 --2- 10.20.193.27:0/50719270 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe754122b00 0x7fe754122fa0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:42.106 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:42.104+0000 7fe759b786c0 1 --2- 10.20.193.27:0/50719270 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fe754115440 0x7fe75411d740 unknown :-1 s=CLOSED pgs=447 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:42.106 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:42.104+0000 7fe759b786c0 1 --2- 10.20.193.27:0/50719270 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fe75410f230 0x7fe75411d200 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:42.106 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:42.104+0000 7fe759b786c0 1 -- 10.20.193.27:0/50719270 >> 10.20.193.27:0/50719270 conn(0x7fe7540ecb30 msgr2=0x7fe75410cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:42.107 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:42.104+0000 7fe759b786c0 1 -- 10.20.193.27:0/50719270 shutdown_connections 2026-02-13T02:25:42.107 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:42.104+0000 7fe759b786c0 1 -- 10.20.193.27:0/50719270 wait complete. 2026-02-13T02:25:43.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:43 trial027 ceph-mon[24567]: pgmap v292: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:43.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:43 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/50719270' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:43.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:43 trial028 ceph-mon[26701]: pgmap v292: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:43.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:43 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/50719270' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:43.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:43 trial107 ceph-mon[27422]: pgmap v292: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:43.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:43 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/50719270' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:45.148 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:25:45.266 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:25:45.380 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.377+0000 7fc4917d46c0 1 -- 10.20.193.27:0/1469218761 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc48c115410 msgr2=0x7fc48c175db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:45.381 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.377+0000 7fc4917d46c0 1 --2- 10.20.193.27:0/1469218761 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc48c115410 0x7fc48c175db0 secure :-1 s=READY pgs=448 gs=1 cs=0 l=1 c_cookie=3072607587512986813 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc47c00aa70 tx=0x7fc47c02fde0 comp rx=0 tx=0).stop 2026-02-13T02:25:45.381 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.377+0000 7fc4917d46c0 1 -- 10.20.193.27:0/1469218761 shutdown_connections 2026-02-13T02:25:45.381 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.377+0000 7fc4917d46c0 1 --2- 10.20.193.27:0/1469218761 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc48c115410 0x7fc48c175db0 unknown :-1 s=CLOSED pgs=448 gs=1 cs=0 l=1 c_cookie=3072607587512986813 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:45.381 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.377+0000 7fc4917d46c0 1 --2- 10.20.193.27:0/1469218761 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc48c10fbf0 0x7fc48c114c30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:45.381 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.377+0000 7fc4917d46c0 1 --2- 10.20.193.27:0/1469218761 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc48c10f200 0x7fc48c10f5d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:45.381 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.377+0000 7fc4917d46c0 1 -- 10.20.193.27:0/1469218761 >> 10.20.193.27:0/1469218761 conn(0x7fc48c0ecb00 msgr2=0x7fc48c10cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:45.381 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.377+0000 7fc4917d46c0 1 -- 10.20.193.27:0/1469218761 shutdown_connections 2026-02-13T02:25:45.381 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.377+0000 7fc4917d46c0 1 -- 10.20.193.27:0/1469218761 wait complete. 2026-02-13T02:25:45.381 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.377+0000 7fc4917d46c0 1 Processor -- start 2026-02-13T02:25:45.382 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:45 trial027 ceph-mon[24567]: pgmap v293: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:45.384 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.377+0000 7fc4917d46c0 1 -- start start 2026-02-13T02:25:45.384 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.378+0000 7fc4917d46c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc48c10f200 0x7fc48c11d1d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:45.384 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.378+0000 7fc4917d46c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc48c115410 0x7fc48c11d710 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:45.385 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.378+0000 7fc4917d46c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc48c122ad0 0x7fc48c122f70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:45.385 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.378+0000 7fc4917d46c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc48c1234e0 con 0x7fc48c10f200 2026-02-13T02:25:45.385 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.378+0000 7fc4917d46c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc48c123650 con 0x7fc48c122ad0 2026-02-13T02:25:45.385 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.378+0000 7fc4917d46c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc48c123810 con 0x7fc48c115410 2026-02-13T02:25:45.385 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.378+0000 7fc48affd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc48c10f200 0x7fc48c11d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:45.385 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.378+0000 7fc48affd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc48c10f200 0x7fc48c11d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:58124/0 (socket says 10.20.193.27:58124) 2026-02-13T02:25:45.385 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.378+0000 7fc48affd6c0 1 -- 10.20.193.27:0/1651805510 learned_addr learned my addr 10.20.193.27:0/1651805510 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:25:45.385 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.378+0000 7fc48b7fe6c0 1 --2- 10.20.193.27:0/1651805510 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc48c122ad0 0x7fc48c122f70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:45.385 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.378+0000 7fc48a7fc6c0 1 --2- 10.20.193.27:0/1651805510 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc48c115410 0x7fc48c11d710 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:45.385 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.378+0000 7fc48affd6c0 1 -- 10.20.193.27:0/1651805510 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc48c115410 msgr2=0x7fc48c11d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:45.385 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.378+0000 7fc48affd6c0 1 --2- 10.20.193.27:0/1651805510 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc48c115410 0x7fc48c11d710 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:45.385 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.378+0000 7fc48affd6c0 1 -- 10.20.193.27:0/1651805510 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc48c122ad0 msgr2=0x7fc48c122f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:45.386 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.378+0000 7fc48affd6c0 1 --2- 10.20.193.27:0/1651805510 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc48c122ad0 0x7fc48c122f70 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:45.386 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.378+0000 7fc48affd6c0 1 -- 10.20.193.27:0/1651805510 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47c00a660 con 0x7fc48c10f200 2026-02-13T02:25:45.386 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.378+0000 7fc48a7fc6c0 1 --2- 10.20.193.27:0/1651805510 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc48c115410 0x7fc48c11d710 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:25:45.386 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.378+0000 7fc48affd6c0 1 --2- 10.20.193.27:0/1651805510 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc48c10f200 0x7fc48c11d1d0 secure :-1 s=READY pgs=449 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc480005dc0 tx=0x7fc48000bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:45.386 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.378+0000 7fc46bfff6c0 1 -- 10.20.193.27:0/1651805510 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc480004250 con 0x7fc48c10f200 2026-02-13T02:25:45.386 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.378+0000 7fc4917d46c0 1 -- 10.20.193.27:0/1651805510 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc48c163cc0 con 0x7fc48c10f200 2026-02-13T02:25:45.386 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.378+0000 7fc46bfff6c0 1 -- 10.20.193.27:0/1651805510 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc480009070 con 0x7fc48c10f200 2026-02-13T02:25:45.386 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.378+0000 7fc46bfff6c0 1 -- 10.20.193.27:0/1651805510 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc48000cd70 con 0x7fc48c10f200 2026-02-13T02:25:45.386 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.378+0000 7fc4917d46c0 1 -- 10.20.193.27:0/1651805510 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc48c164150 con 0x7fc48c10f200 2026-02-13T02:25:45.386 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.378+0000 7fc4917d46c0 1 -- 10.20.193.27:0/1651805510 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc48c1104d0 con 0x7fc48c10f200 2026-02-13T02:25:45.386 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.380+0000 7fc46bfff6c0 1 -- 10.20.193.27:0/1651805510 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7fc4800028c0 con 0x7fc48c10f200 2026-02-13T02:25:45.386 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.380+0000 7fc46bfff6c0 1 --2- 10.20.193.27:0/1651805510 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc4600b5600 0x7fc4600d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:45.387 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.380+0000 7fc46bfff6c0 1 -- 10.20.193.27:0/1651805510 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7fc4800e3b70 con 0x7fc48c10f200 2026-02-13T02:25:45.387 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.380+0000 7fc48a7fc6c0 1 --2- 10.20.193.27:0/1651805510 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc4600b5600 0x7fc4600d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:45.387 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.381+0000 7fc46bfff6c0 1 -- 10.20.193.27:0/1651805510 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fc4800a65f0 con 0x7fc48c10f200 2026-02-13T02:25:45.387 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.381+0000 7fc48a7fc6c0 1 --2- 10.20.193.27:0/1651805510 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc4600b5600 0x7fc4600d5ab0 secure :-1 s=READY pgs=288 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc48c123b60 tx=0x7fc474005f10 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:45.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:45 trial028 ceph-mon[26701]: pgmap v293: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:45.496 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.493+0000 7fc4917d46c0 1 -- 10.20.193.27:0/1651805510 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc48c10f5d0 con 0x7fc48c10f200 2026-02-13T02:25:45.496 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.494+0000 7fc46bfff6c0 1 -- 10.20.193.27:0/1651805510 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7fc4800a97a0 con 0x7fc48c10f200 2026-02-13T02:25:45.496 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:25:45.496 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:25:45.497 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.494+0000 7fc4917d46c0 1 -- 10.20.193.27:0/1651805510 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc4600b5600 msgr2=0x7fc4600d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:45.498 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.494+0000 7fc4917d46c0 1 --2- 10.20.193.27:0/1651805510 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc4600b5600 0x7fc4600d5ab0 secure :-1 s=READY pgs=288 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc48c123b60 tx=0x7fc474005f10 comp rx=0 tx=0).stop 2026-02-13T02:25:45.498 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.494+0000 7fc4917d46c0 1 -- 10.20.193.27:0/1651805510 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc48c10f200 msgr2=0x7fc48c11d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:45.498 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.494+0000 7fc4917d46c0 1 --2- 10.20.193.27:0/1651805510 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc48c10f200 0x7fc48c11d1d0 secure :-1 s=READY pgs=449 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc480005dc0 tx=0x7fc48000bbb0 comp rx=0 tx=0).stop 2026-02-13T02:25:45.498 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.495+0000 7fc4917d46c0 1 -- 10.20.193.27:0/1651805510 shutdown_connections 2026-02-13T02:25:45.498 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.495+0000 7fc4917d46c0 1 --2- 10.20.193.27:0/1651805510 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc4600b5600 0x7fc4600d5ab0 unknown :-1 s=CLOSED pgs=288 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:45.498 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.495+0000 7fc4917d46c0 1 --2- 10.20.193.27:0/1651805510 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc48c122ad0 0x7fc48c122f70 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:45.498 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.495+0000 7fc4917d46c0 1 --2- 10.20.193.27:0/1651805510 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc48c115410 0x7fc48c11d710 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:45.498 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.495+0000 7fc4917d46c0 1 --2- 10.20.193.27:0/1651805510 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc48c10f200 0x7fc48c11d1d0 unknown :-1 s=CLOSED pgs=449 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:45.498 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.495+0000 7fc4917d46c0 1 -- 10.20.193.27:0/1651805510 >> 10.20.193.27:0/1651805510 conn(0x7fc48c0ecb00 msgr2=0x7fc48c10cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:45.498 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.495+0000 7fc4917d46c0 1 -- 10.20.193.27:0/1651805510 shutdown_connections 2026-02-13T02:25:45.499 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:45.495+0000 7fc4917d46c0 1 -- 10.20.193.27:0/1651805510 wait complete. 2026-02-13T02:25:45.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:45 trial107 ceph-mon[27422]: pgmap v293: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:46.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:46 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1651805510' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:46.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:46 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1651805510' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:46.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:46 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1651805510' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:47.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:47 trial027 ceph-mon[24567]: pgmap v294: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:47.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:47 trial028 ceph-mon[26701]: pgmap v294: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:47.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:47 trial107 ceph-mon[27422]: pgmap v294: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:48.539 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:25:48.658 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:25:48.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.770+0000 7f46d16756c0 1 -- 10.20.193.27:0/1745811239 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f46cc10fc20 msgr2=0x7f46cc114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:48.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.770+0000 7f46d16756c0 1 --2- 10.20.193.27:0/1745811239 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f46cc10fc20 0x7f46cc114c60 secure :-1 s=READY pgs=450 gs=1 cs=0 l=1 c_cookie=2987922775469149675 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f46b40099a0 tx=0x7f46b402ed20 comp rx=0 tx=0).stop 2026-02-13T02:25:48.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.770+0000 7f46d16756c0 1 -- 10.20.193.27:0/1745811239 shutdown_connections 2026-02-13T02:25:48.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.770+0000 7f46d16756c0 1 --2- 10.20.193.27:0/1745811239 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f46cc115440 0x7f46cc175de0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:48.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.770+0000 7f46d16756c0 1 --2- 10.20.193.27:0/1745811239 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f46cc10fc20 0x7f46cc114c60 unknown :-1 s=CLOSED pgs=450 gs=1 cs=0 l=1 c_cookie=2987922775469149675 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:48.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.770+0000 7f46d16756c0 1 --2- 10.20.193.27:0/1745811239 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f46cc10f230 0x7f46cc10f600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:48.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.770+0000 7f46d16756c0 1 -- 10.20.193.27:0/1745811239 >> 10.20.193.27:0/1745811239 conn(0x7f46cc0ecb30 msgr2=0x7f46cc10cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:48.775 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.770+0000 7f46d16756c0 1 -- 10.20.193.27:0/1745811239 shutdown_connections 2026-02-13T02:25:48.776 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.770+0000 7f46d16756c0 1 -- 10.20.193.27:0/1745811239 wait complete. 2026-02-13T02:25:48.776 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.770+0000 7f46d16756c0 1 Processor -- start 2026-02-13T02:25:48.776 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.770+0000 7f46d16756c0 1 -- start start 2026-02-13T02:25:48.776 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.771+0000 7f46d16756c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f46cc10f230 0x7f46cc0f4cc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:48.776 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.771+0000 7f46d16756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f46cc115440 0x7f46cc175ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:48.776 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.771+0000 7f46d16756c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f46cc0f52f0 0x7f46cc0fa5c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:48.776 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.771+0000 7f46d16756c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc0f59e0 con 0x7f46cc115440 2026-02-13T02:25:48.776 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.771+0000 7f46d16756c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc0fab00 con 0x7f46cc10f230 2026-02-13T02:25:48.776 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.771+0000 7f46d16756c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc0facc0 con 0x7f46cc0f52f0 2026-02-13T02:25:48.776 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.771+0000 7f46ca7fc6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f46cc115440 0x7f46cc175ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:48.776 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.771+0000 7f46ca7fc6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f46cc115440 0x7f46cc175ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:34642/0 (socket says 10.20.193.27:34642) 2026-02-13T02:25:48.777 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.771+0000 7f46ca7fc6c0 1 -- 10.20.193.27:0/435237450 learned_addr learned my addr 10.20.193.27:0/435237450 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:25:48.777 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.771+0000 7f46caffd6c0 1 --2- 10.20.193.27:0/435237450 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f46cc10f230 0x7f46cc0f4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:48.777 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.771+0000 7f46cb7fe6c0 1 --2- 10.20.193.27:0/435237450 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f46cc0f52f0 0x7f46cc0fa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:48.777 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.771+0000 7f46ca7fc6c0 1 -- 10.20.193.27:0/435237450 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f46cc0f52f0 msgr2=0x7f46cc0fa5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:48.777 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.771+0000 7f46ca7fc6c0 1 --2- 10.20.193.27:0/435237450 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f46cc0f52f0 0x7f46cc0fa5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:48.777 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.771+0000 7f46ca7fc6c0 1 -- 10.20.193.27:0/435237450 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f46cc10f230 msgr2=0x7f46cc0f4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:48.777 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.771+0000 7f46ca7fc6c0 1 --2- 10.20.193.27:0/435237450 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f46cc10f230 0x7f46cc0f4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:48.777 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.771+0000 7f46ca7fc6c0 1 -- 10.20.193.27:0/435237450 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b4009590 con 0x7f46cc115440 2026-02-13T02:25:48.777 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.771+0000 7f46cb7fe6c0 1 --2- 10.20.193.27:0/435237450 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f46cc0f52f0 0x7f46cc0fa5c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:25:48.777 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.771+0000 7f46ca7fc6c0 1 --2- 10.20.193.27:0/435237450 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f46cc115440 0x7f46cc175ce0 secure :-1 s=READY pgs=451 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f46b4004830 tx=0x7f46b402fbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:48.777 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.771+0000 7f46abfff6c0 1 -- 10.20.193.27:0/435237450 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f46b4002ae0 con 0x7f46cc115440 2026-02-13T02:25:48.777 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.771+0000 7f46abfff6c0 1 -- 10.20.193.27:0/435237450 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f46b4002c40 con 0x7f46cc115440 2026-02-13T02:25:48.777 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.771+0000 7f46d16756c0 1 -- 10.20.193.27:0/435237450 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46cc0faf40 con 0x7f46cc115440 2026-02-13T02:25:48.778 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.771+0000 7f46abfff6c0 1 -- 10.20.193.27:0/435237450 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f46b40362f0 con 0x7f46cc115440 2026-02-13T02:25:48.778 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.771+0000 7f46d16756c0 1 -- 10.20.193.27:0/435237450 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46cc0fb3e0 con 0x7f46cc115440 2026-02-13T02:25:48.778 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.771+0000 7f46d16756c0 1 -- 10.20.193.27:0/435237450 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46cc110500 con 0x7f46cc115440 2026-02-13T02:25:48.778 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.773+0000 7f46abfff6c0 1 -- 10.20.193.27:0/435237450 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f46b4007810 con 0x7f46cc115440 2026-02-13T02:25:48.779 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.773+0000 7f46abfff6c0 1 --2- 10.20.193.27:0/435237450 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f46a00b5680 0x7f46a00d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:48.779 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.773+0000 7f46abfff6c0 1 -- 10.20.193.27:0/435237450 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f46b4107ba0 con 0x7f46cc115440 2026-02-13T02:25:48.779 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.773+0000 7f46caffd6c0 1 --2- 10.20.193.27:0/435237450 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f46a00b5680 0x7f46a00d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:48.779 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.774+0000 7f46caffd6c0 1 --2- 10.20.193.27:0/435237450 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f46a00b5680 0x7f46a00d5b30 secure :-1 s=READY pgs=289 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f46c0005980 tx=0x7f46c0005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:48.779 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.774+0000 7f46abfff6c0 1 -- 10.20.193.27:0/435237450 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f46b40ca620 con 0x7f46cc115440 2026-02-13T02:25:48.894 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.891+0000 7f46d16756c0 1 -- 10.20.193.27:0/435237450 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f46cc10fc20 con 0x7f46cc115440 2026-02-13T02:25:48.894 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.891+0000 7f46abfff6c0 1 -- 10.20.193.27:0/435237450 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f46b40cd7d0 con 0x7f46cc115440 2026-02-13T02:25:48.894 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:25:48.894 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:25:48.895 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.893+0000 7f46d16756c0 1 -- 10.20.193.27:0/435237450 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f46a00b5680 msgr2=0x7f46a00d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:48.895 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.893+0000 7f46d16756c0 1 --2- 10.20.193.27:0/435237450 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f46a00b5680 0x7f46a00d5b30 secure :-1 s=READY pgs=289 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f46c0005980 tx=0x7f46c0005910 comp rx=0 tx=0).stop 2026-02-13T02:25:48.896 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.893+0000 7f46d16756c0 1 -- 10.20.193.27:0/435237450 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f46cc115440 msgr2=0x7f46cc175ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:48.896 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.893+0000 7f46d16756c0 1 --2- 10.20.193.27:0/435237450 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f46cc115440 0x7f46cc175ce0 secure :-1 s=READY pgs=451 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f46b4004830 tx=0x7f46b402fbf0 comp rx=0 tx=0).stop 2026-02-13T02:25:48.896 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.893+0000 7f46d16756c0 1 -- 10.20.193.27:0/435237450 shutdown_connections 2026-02-13T02:25:48.896 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.893+0000 7f46d16756c0 1 --2- 10.20.193.27:0/435237450 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f46a00b5680 0x7f46a00d5b30 unknown :-1 s=CLOSED pgs=289 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:48.896 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.893+0000 7f46d16756c0 1 --2- 10.20.193.27:0/435237450 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f46cc0f52f0 0x7f46cc0fa5c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:48.896 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.893+0000 7f46d16756c0 1 --2- 10.20.193.27:0/435237450 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f46cc115440 0x7f46cc175ce0 unknown :-1 s=CLOSED pgs=451 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:48.896 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.893+0000 7f46d16756c0 1 --2- 10.20.193.27:0/435237450 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f46cc10f230 0x7f46cc0f4cc0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:48.896 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.893+0000 7f46d16756c0 1 -- 10.20.193.27:0/435237450 >> 10.20.193.27:0/435237450 conn(0x7f46cc0ecb30 msgr2=0x7f46cc10d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:48.896 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.893+0000 7f46d16756c0 1 -- 10.20.193.27:0/435237450 shutdown_connections 2026-02-13T02:25:48.896 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:48.893+0000 7f46d16756c0 1 -- 10.20.193.27:0/435237450 wait complete. 2026-02-13T02:25:49.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:49 trial027 ceph-mon[24567]: pgmap v295: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:49.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:49 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/435237450' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:49.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:49 trial028 ceph-mon[26701]: pgmap v295: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:49.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:49 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/435237450' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:49.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:49 trial107 ceph-mon[27422]: pgmap v295: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:49.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:49 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/435237450' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:51.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:51 trial027 ceph-mon[24567]: pgmap v296: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:51.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:51 trial028 ceph-mon[26701]: pgmap v296: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:51.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:51 trial107 ceph-mon[27422]: pgmap v296: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:51.937 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:25:52.060 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:25:52.177 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.173+0000 7f7b9bf4c6c0 1 -- 10.20.193.27:0/606824828 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7b9410cfb0 msgr2=0x7f7b9410d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:52.177 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.173+0000 7f7b9bf4c6c0 1 --2- 10.20.193.27:0/606824828 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7b9410cfb0 0x7f7b9410d380 secure :-1 s=READY pgs=452 gs=3 cs=0 l=1 c_cookie=17318113169531490657 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7b840099a0 tx=0x7f7b8402ed50 comp rx=0 tx=0).stop 2026-02-13T02:25:52.178 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.173+0000 7f7b9bf4c6c0 1 -- 10.20.193.27:0/606824828 shutdown_connections 2026-02-13T02:25:52.178 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.173+0000 7f7b9bf4c6c0 1 --2- 10.20.193.27:0/606824828 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f7b941131c0 0x7f7b94173b60 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:52.178 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.173+0000 7f7b9bf4c6c0 1 --2- 10.20.193.27:0/606824828 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7b9410d9a0 0x7f7b941129e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:52.178 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.173+0000 7f7b9bf4c6c0 1 --2- 10.20.193.27:0/606824828 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7b9410cfb0 0x7f7b9410d380 unknown :-1 s=CLOSED pgs=452 gs=3 cs=0 l=1 c_cookie=17318113169531490657 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:52.178 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.173+0000 7f7b9bf4c6c0 1 -- 10.20.193.27:0/606824828 >> 10.20.193.27:0/606824828 conn(0x7f7b940ea8b0 msgr2=0x7f7b9410acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:52.178 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.173+0000 7f7b9bf4c6c0 1 -- 10.20.193.27:0/606824828 shutdown_connections 2026-02-13T02:25:52.178 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.173+0000 7f7b9bf4c6c0 1 -- 10.20.193.27:0/606824828 wait complete. 2026-02-13T02:25:52.178 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.173+0000 7f7b9bf4c6c0 1 Processor -- start 2026-02-13T02:25:52.178 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.173+0000 7f7b9bf4c6c0 1 -- start start 2026-02-13T02:25:52.178 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.173+0000 7f7b9bf4c6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7b9410cfb0 0x7f7b940f4c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:52.179 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.173+0000 7f7b9bf4c6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f7b941131c0 0x7f7b94173a60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:52.179 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.173+0000 7f7b9bf4c6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7b940f52c0 0x7f7b940fa590 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:52.179 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.173+0000 7f7b9bf4c6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7b940f5920 con 0x7f7b940f52c0 2026-02-13T02:25:52.179 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.173+0000 7f7b9bf4c6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7b940faad0 con 0x7f7b9410cfb0 2026-02-13T02:25:52.179 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.173+0000 7f7b9bf4c6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7b940fac90 con 0x7f7b941131c0 2026-02-13T02:25:52.179 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.174+0000 7f7b9a4be6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7b940f52c0 0x7f7b940fa590 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:52.179 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.174+0000 7f7b9a4be6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7b940f52c0 0x7f7b940fa590 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:34652/0 (socket says 10.20.193.27:34652) 2026-02-13T02:25:52.179 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.174+0000 7f7b9a4be6c0 1 -- 10.20.193.27:0/2931802831 learned_addr learned my addr 10.20.193.27:0/2931802831 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:25:52.179 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.174+0000 7f7b994bc6c0 1 --2- 10.20.193.27:0/2931802831 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f7b941131c0 0x7f7b94173a60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:52.179 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.174+0000 7f7b99cbd6c0 1 --2- 10.20.193.27:0/2931802831 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7b9410cfb0 0x7f7b940f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:52.179 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.174+0000 7f7b9a4be6c0 1 -- 10.20.193.27:0/2931802831 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f7b941131c0 msgr2=0x7f7b94173a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:52.179 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.174+0000 7f7b9a4be6c0 1 --2- 10.20.193.27:0/2931802831 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f7b941131c0 0x7f7b94173a60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:52.180 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.174+0000 7f7b9a4be6c0 1 -- 10.20.193.27:0/2931802831 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7b9410cfb0 msgr2=0x7f7b940f4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:52.180 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.174+0000 7f7b9a4be6c0 1 --2- 10.20.193.27:0/2931802831 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7b9410cfb0 0x7f7b940f4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:52.180 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.174+0000 7f7b9a4be6c0 1 -- 10.20.193.27:0/2931802831 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b84009590 con 0x7f7b940f52c0 2026-02-13T02:25:52.180 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.174+0000 7f7b994bc6c0 1 --2- 10.20.193.27:0/2931802831 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f7b941131c0 0x7f7b94173a60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:25:52.180 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.174+0000 7f7b99cbd6c0 1 --2- 10.20.193.27:0/2931802831 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7b9410cfb0 0x7f7b940f4c90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:25:52.180 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.174+0000 7f7b9a4be6c0 1 --2- 10.20.193.27:0/2931802831 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7b940f52c0 0x7f7b940fa590 secure :-1 s=READY pgs=453 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7b9000c730 tx=0x7f7b9000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:52.180 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.174+0000 7f7b82ffd6c0 1 -- 10.20.193.27:0/2931802831 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f7b90003c40 con 0x7f7b940f52c0 2026-02-13T02:25:52.180 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.174+0000 7f7b82ffd6c0 1 -- 10.20.193.27:0/2931802831 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f7b9000a070 con 0x7f7b940f52c0 2026-02-13T02:25:52.180 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.174+0000 7f7b9bf4c6c0 1 -- 10.20.193.27:0/2931802831 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b940faf10 con 0x7f7b940f52c0 2026-02-13T02:25:52.180 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.174+0000 7f7b82ffd6c0 1 -- 10.20.193.27:0/2931802831 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f7b9000dca0 con 0x7f7b940f52c0 2026-02-13T02:25:52.180 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.174+0000 7f7b9bf4c6c0 1 -- 10.20.193.27:0/2931802831 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b940fb3f0 con 0x7f7b940f52c0 2026-02-13T02:25:52.180 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.174+0000 7f7b9bf4c6c0 1 -- 10.20.193.27:0/2931802831 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b9410e280 con 0x7f7b940f52c0 2026-02-13T02:25:52.182 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.176+0000 7f7b82ffd6c0 1 -- 10.20.193.27:0/2931802831 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f7b90005030 con 0x7f7b940f52c0 2026-02-13T02:25:52.182 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.176+0000 7f7b82ffd6c0 1 --2- 10.20.193.27:0/2931802831 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f7b700b5600 0x7f7b700d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:52.182 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.176+0000 7f7b82ffd6c0 1 -- 10.20.193.27:0/2931802831 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f7b900e4ad0 con 0x7f7b940f52c0 2026-02-13T02:25:52.182 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.176+0000 7f7b99cbd6c0 1 --2- 10.20.193.27:0/2931802831 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f7b700b5600 0x7f7b700d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:52.182 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.176+0000 7f7b82ffd6c0 1 -- 10.20.193.27:0/2931802831 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f7b900a7590 con 0x7f7b940f52c0 2026-02-13T02:25:52.182 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.176+0000 7f7b99cbd6c0 1 --2- 10.20.193.27:0/2931802831 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f7b700b5600 0x7f7b700d5ab0 secure :-1 s=READY pgs=290 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7b84009970 tx=0x7f7b840023a0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:52.294 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.291+0000 7f7b9bf4c6c0 1 -- 10.20.193.27:0/2931802831 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f7b9410d380 con 0x7f7b940f52c0 2026-02-13T02:25:52.294 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.292+0000 7f7b82ffd6c0 1 -- 10.20.193.27:0/2931802831 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f7b90005310 con 0x7f7b940f52c0 2026-02-13T02:25:52.295 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:25:52.295 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:25:52.296 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.293+0000 7f7b9bf4c6c0 1 -- 10.20.193.27:0/2931802831 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f7b700b5600 msgr2=0x7f7b700d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:52.296 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.293+0000 7f7b9bf4c6c0 1 --2- 10.20.193.27:0/2931802831 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f7b700b5600 0x7f7b700d5ab0 secure :-1 s=READY pgs=290 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7b84009970 tx=0x7f7b840023a0 comp rx=0 tx=0).stop 2026-02-13T02:25:52.296 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.293+0000 7f7b9bf4c6c0 1 -- 10.20.193.27:0/2931802831 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7b940f52c0 msgr2=0x7f7b940fa590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:52.296 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.293+0000 7f7b9bf4c6c0 1 --2- 10.20.193.27:0/2931802831 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7b940f52c0 0x7f7b940fa590 secure :-1 s=READY pgs=453 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7b9000c730 tx=0x7f7b9000cc00 comp rx=0 tx=0).stop 2026-02-13T02:25:52.296 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.293+0000 7f7b9bf4c6c0 1 -- 10.20.193.27:0/2931802831 shutdown_connections 2026-02-13T02:25:52.296 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.293+0000 7f7b9bf4c6c0 1 --2- 10.20.193.27:0/2931802831 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f7b700b5600 0x7f7b700d5ab0 unknown :-1 s=CLOSED pgs=290 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:52.297 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.293+0000 7f7b9bf4c6c0 1 --2- 10.20.193.27:0/2931802831 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f7b940f52c0 0x7f7b940fa590 unknown :-1 s=CLOSED pgs=453 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:52.297 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.293+0000 7f7b9bf4c6c0 1 --2- 10.20.193.27:0/2931802831 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f7b941131c0 0x7f7b94173a60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:52.297 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.293+0000 7f7b9bf4c6c0 1 --2- 10.20.193.27:0/2931802831 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7b9410cfb0 0x7f7b940f4c90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:52.297 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.293+0000 7f7b9bf4c6c0 1 -- 10.20.193.27:0/2931802831 >> 10.20.193.27:0/2931802831 conn(0x7f7b940ea8b0 msgr2=0x7f7b940605e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:52.297 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.293+0000 7f7b9bf4c6c0 1 -- 10.20.193.27:0/2931802831 shutdown_connections 2026-02-13T02:25:52.297 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:52.293+0000 7f7b9bf4c6c0 1 -- 10.20.193.27:0/2931802831 wait complete. 2026-02-13T02:25:53.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:53 trial027 ceph-mon[24567]: pgmap v297: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:53.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:53 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2931802831' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:53.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:53 trial028 ceph-mon[26701]: pgmap v297: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:53.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:53 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2931802831' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:53.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:53 trial107 ceph-mon[27422]: pgmap v297: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:53.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:53 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2931802831' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:55.338 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:25:55.363 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:55 trial027 ceph-mon[24567]: pgmap v298: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:55.457 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:25:55.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:55 trial028 ceph-mon[26701]: pgmap v298: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:55.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:55 trial107 ceph-mon[27422]: pgmap v298: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:55.574 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.570+0000 7f10cfcbb6c0 1 -- 10.20.193.27:0/3762598324 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f10c810f200 msgr2=0x7f10c810f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:55.575 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.570+0000 7f10cfcbb6c0 1 --2- 10.20.193.27:0/3762598324 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f10c810f200 0x7f10c810f5d0 secure :-1 s=READY pgs=454 gs=1 cs=0 l=1 c_cookie=7645313131205400621 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f10b80099a0 tx=0x7f10b802ed20 comp rx=0 tx=0).stop 2026-02-13T02:25:55.575 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.570+0000 7f10cfcbb6c0 1 -- 10.20.193.27:0/3762598324 shutdown_connections 2026-02-13T02:25:55.575 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.570+0000 7f10cfcbb6c0 1 --2- 10.20.193.27:0/3762598324 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f10c8115410 0x7f10c8175db0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:55.575 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.570+0000 7f10cfcbb6c0 1 --2- 10.20.193.27:0/3762598324 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f10c810fbf0 0x7f10c8114c30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:55.575 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.570+0000 7f10cfcbb6c0 1 --2- 10.20.193.27:0/3762598324 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f10c810f200 0x7f10c810f5d0 unknown :-1 s=CLOSED pgs=454 gs=1 cs=0 l=1 c_cookie=7645313131205400621 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:55.575 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.570+0000 7f10cfcbb6c0 1 -- 10.20.193.27:0/3762598324 >> 10.20.193.27:0/3762598324 conn(0x7f10c80ecb00 msgr2=0x7f10c810cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:55.575 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.570+0000 7f10cfcbb6c0 1 -- 10.20.193.27:0/3762598324 shutdown_connections 2026-02-13T02:25:55.575 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.570+0000 7f10cfcbb6c0 1 -- 10.20.193.27:0/3762598324 wait complete. 2026-02-13T02:25:55.575 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.570+0000 7f10cfcbb6c0 1 Processor -- start 2026-02-13T02:25:55.576 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.570+0000 7f10cfcbb6c0 1 -- start start 2026-02-13T02:25:55.576 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.571+0000 7f10cfcbb6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f10c810f200 0x7f10c811d2e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:55.576 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.571+0000 7f10cfcbb6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f10c8115410 0x7f10c811d820 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:55.576 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.571+0000 7f10cfcbb6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f10c8122be0 0x7f10c8123080 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:55.576 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.571+0000 7f10cfcbb6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f10c81235f0 con 0x7f10c8115410 2026-02-13T02:25:55.576 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.571+0000 7f10cfcbb6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f10c8123760 con 0x7f10c8122be0 2026-02-13T02:25:55.576 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.571+0000 7f10cfcbb6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f10c8123920 con 0x7f10c810f200 2026-02-13T02:25:55.576 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.571+0000 7f10cd22b6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f10c8115410 0x7f10c811d820 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:55.576 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.571+0000 7f10cd22b6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f10c8115410 0x7f10c811d820 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:34680/0 (socket says 10.20.193.27:34680) 2026-02-13T02:25:55.576 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.571+0000 7f10cd22b6c0 1 -- 10.20.193.27:0/3355297583 learned_addr learned my addr 10.20.193.27:0/3355297583 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:25:55.576 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.571+0000 7f10cda2c6c0 1 --2- 10.20.193.27:0/3355297583 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f10c810f200 0x7f10c811d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:55.576 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.571+0000 7f10ce22d6c0 1 --2- 10.20.193.27:0/3355297583 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f10c8122be0 0x7f10c8123080 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:55.577 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.571+0000 7f10cd22b6c0 1 -- 10.20.193.27:0/3355297583 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f10c810f200 msgr2=0x7f10c811d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:55.577 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.571+0000 7f10cd22b6c0 1 --2- 10.20.193.27:0/3355297583 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f10c810f200 0x7f10c811d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:55.577 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.571+0000 7f10cd22b6c0 1 -- 10.20.193.27:0/3355297583 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f10c8122be0 msgr2=0x7f10c8123080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:55.577 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.571+0000 7f10cd22b6c0 1 --2- 10.20.193.27:0/3355297583 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f10c8122be0 0x7f10c8123080 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:55.577 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.571+0000 7f10cd22b6c0 1 -- 10.20.193.27:0/3355297583 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10b8009590 con 0x7f10c8115410 2026-02-13T02:25:55.577 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.571+0000 7f10cda2c6c0 1 --2- 10.20.193.27:0/3355297583 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f10c810f200 0x7f10c811d2e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:25:55.577 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.571+0000 7f10cd22b6c0 1 --2- 10.20.193.27:0/3355297583 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f10c8115410 0x7f10c811d820 secure :-1 s=READY pgs=455 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f10bc005dc0 tx=0x7f10bc00bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:55.577 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.571+0000 7f10b6ffd6c0 1 -- 10.20.193.27:0/3355297583 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f10bc004250 con 0x7f10c8115410 2026-02-13T02:25:55.577 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.571+0000 7f10b6ffd6c0 1 -- 10.20.193.27:0/3355297583 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f10bc009070 con 0x7f10c8115410 2026-02-13T02:25:55.577 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.571+0000 7f10cfcbb6c0 1 -- 10.20.193.27:0/3355297583 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c8163dd0 con 0x7f10c8115410 2026-02-13T02:25:55.577 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.571+0000 7f10b6ffd6c0 1 -- 10.20.193.27:0/3355297583 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f10bc00cd50 con 0x7f10c8115410 2026-02-13T02:25:55.577 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.571+0000 7f10cfcbb6c0 1 -- 10.20.193.27:0/3355297583 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c8164260 con 0x7f10c8115410 2026-02-13T02:25:55.578 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.571+0000 7f10cfcbb6c0 1 -- 10.20.193.27:0/3355297583 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10c81104d0 con 0x7f10c8115410 2026-02-13T02:25:55.578 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.573+0000 7f10b6ffd6c0 1 -- 10.20.193.27:0/3355297583 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f10bc0028c0 con 0x7f10c8115410 2026-02-13T02:25:55.579 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.574+0000 7f10b6ffd6c0 1 --2- 10.20.193.27:0/3355297583 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f10a40b5600 0x7f10a40d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:55.579 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.574+0000 7f10b6ffd6c0 1 -- 10.20.193.27:0/3355297583 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f10bc0e3ae0 con 0x7f10c8115410 2026-02-13T02:25:55.579 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.574+0000 7f10cda2c6c0 1 --2- 10.20.193.27:0/3355297583 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f10a40b5600 0x7f10a40d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:55.579 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.574+0000 7f10b6ffd6c0 1 -- 10.20.193.27:0/3355297583 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f10bc11c5e0 con 0x7f10c8115410 2026-02-13T02:25:55.579 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.574+0000 7f10cda2c6c0 1 --2- 10.20.193.27:0/3355297583 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f10a40b5600 0x7f10a40d5ab0 secure :-1 s=READY pgs=291 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f10b8005b40 tx=0x7f10b8005c00 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:55.689 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.687+0000 7f10cfcbb6c0 1 -- 10.20.193.27:0/3355297583 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f10c810f5d0 con 0x7f10c8115410 2026-02-13T02:25:55.689 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.687+0000 7f10b6ffd6c0 1 -- 10.20.193.27:0/3355297583 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f10bc0a65a0 con 0x7f10c8115410 2026-02-13T02:25:55.689 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:25:55.689 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:25:55.690 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.688+0000 7f10cfcbb6c0 1 -- 10.20.193.27:0/3355297583 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f10a40b5600 msgr2=0x7f10a40d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:55.690 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.688+0000 7f10cfcbb6c0 1 --2- 10.20.193.27:0/3355297583 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f10a40b5600 0x7f10a40d5ab0 secure :-1 s=READY pgs=291 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f10b8005b40 tx=0x7f10b8005c00 comp rx=0 tx=0).stop 2026-02-13T02:25:55.690 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.688+0000 7f10cfcbb6c0 1 -- 10.20.193.27:0/3355297583 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f10c8115410 msgr2=0x7f10c811d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:55.690 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.688+0000 7f10cfcbb6c0 1 --2- 10.20.193.27:0/3355297583 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f10c8115410 0x7f10c811d820 secure :-1 s=READY pgs=455 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f10bc005dc0 tx=0x7f10bc00bbb0 comp rx=0 tx=0).stop 2026-02-13T02:25:55.690 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.688+0000 7f10cfcbb6c0 1 -- 10.20.193.27:0/3355297583 shutdown_connections 2026-02-13T02:25:55.690 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.688+0000 7f10cfcbb6c0 1 --2- 10.20.193.27:0/3355297583 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f10a40b5600 0x7f10a40d5ab0 unknown :-1 s=CLOSED pgs=291 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:55.690 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.688+0000 7f10cfcbb6c0 1 --2- 10.20.193.27:0/3355297583 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f10c8122be0 0x7f10c8123080 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:55.690 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.688+0000 7f10cfcbb6c0 1 --2- 10.20.193.27:0/3355297583 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f10c8115410 0x7f10c811d820 unknown :-1 s=CLOSED pgs=455 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:55.690 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.688+0000 7f10cfcbb6c0 1 --2- 10.20.193.27:0/3355297583 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f10c810f200 0x7f10c811d2e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:55.691 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.688+0000 7f10cfcbb6c0 1 -- 10.20.193.27:0/3355297583 >> 10.20.193.27:0/3355297583 conn(0x7f10c80ecb00 msgr2=0x7f10c810cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:55.691 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.688+0000 7f10cfcbb6c0 1 -- 10.20.193.27:0/3355297583 shutdown_connections 2026-02-13T02:25:55.691 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:55.688+0000 7f10cfcbb6c0 1 -- 10.20.193.27:0/3355297583 wait complete. 2026-02-13T02:25:56.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:56 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3355297583' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:56.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:56 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3355297583' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:56.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:56 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3355297583' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:57.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:57 trial027 ceph-mon[24567]: pgmap v299: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:57.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:57 trial028 ceph-mon[26701]: pgmap v299: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:57.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:57 trial107 ceph-mon[27422]: pgmap v299: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:25:58.732 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:25:58.851 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:25:58.971 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.966+0000 7f3c8a8a46c0 1 -- 10.20.193.27:0/3414663510 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3c8410f200 msgr2=0x7f3c8410f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:58.971 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.966+0000 7f3c8a8a46c0 1 --2- 10.20.193.27:0/3414663510 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3c8410f200 0x7f3c8410f5d0 secure :-1 s=READY pgs=456 gs=1 cs=0 l=1 c_cookie=16495447229203056870 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3c700099a0 tx=0x7f3c7002ed20 comp rx=0 tx=0).stop 2026-02-13T02:25:58.971 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.966+0000 7f3c8a8a46c0 1 -- 10.20.193.27:0/3414663510 shutdown_connections 2026-02-13T02:25:58.971 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.966+0000 7f3c8a8a46c0 1 --2- 10.20.193.27:0/3414663510 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3c84115410 0x7f3c84175db0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:58.972 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.966+0000 7f3c8a8a46c0 1 --2- 10.20.193.27:0/3414663510 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3c8410fbf0 0x7f3c84114c30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:58.972 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.966+0000 7f3c8a8a46c0 1 --2- 10.20.193.27:0/3414663510 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3c8410f200 0x7f3c8410f5d0 unknown :-1 s=CLOSED pgs=456 gs=1 cs=0 l=1 c_cookie=16495447229203056870 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:58.972 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.966+0000 7f3c8a8a46c0 1 -- 10.20.193.27:0/3414663510 >> 10.20.193.27:0/3414663510 conn(0x7f3c840ecb00 msgr2=0x7f3c8410cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:58.972 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.967+0000 7f3c8a8a46c0 1 -- 10.20.193.27:0/3414663510 shutdown_connections 2026-02-13T02:25:58.972 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.967+0000 7f3c8a8a46c0 1 -- 10.20.193.27:0/3414663510 wait complete. 2026-02-13T02:25:58.972 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.967+0000 7f3c8a8a46c0 1 Processor -- start 2026-02-13T02:25:58.972 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.967+0000 7f3c8a8a46c0 1 -- start start 2026-02-13T02:25:58.972 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.967+0000 7f3c8a8a46c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3c8410f200 0x7f3c8411d350 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:58.972 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.967+0000 7f3c8a8a46c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3c84115410 0x7f3c8411d890 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:58.972 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.967+0000 7f3c8a8a46c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3c84122c50 0x7f3c841230f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:58.972 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.967+0000 7f3c8a8a46c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3c84123660 con 0x7f3c8410f200 2026-02-13T02:25:58.972 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.967+0000 7f3c8a8a46c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3c841237d0 con 0x7f3c84115410 2026-02-13T02:25:58.973 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.967+0000 7f3c8a8a46c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3c84123990 con 0x7f3c84122c50 2026-02-13T02:25:58.973 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.967+0000 7f3c83fff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3c8410f200 0x7f3c8411d350 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:58.973 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.967+0000 7f3c83fff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3c8410f200 0x7f3c8411d350 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:46028/0 (socket says 10.20.193.27:46028) 2026-02-13T02:25:58.973 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.967+0000 7f3c83fff6c0 1 -- 10.20.193.27:0/2994359077 learned_addr learned my addr 10.20.193.27:0/2994359077 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:25:58.973 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.967+0000 7f3c837fe6c0 1 --2- 10.20.193.27:0/2994359077 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3c84115410 0x7f3c8411d890 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:58.973 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.967+0000 7f3c88e166c0 1 --2- 10.20.193.27:0/2994359077 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3c84122c50 0x7f3c841230f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:58.973 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.967+0000 7f3c83fff6c0 1 -- 10.20.193.27:0/2994359077 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3c84122c50 msgr2=0x7f3c841230f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:58.973 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.967+0000 7f3c83fff6c0 1 --2- 10.20.193.27:0/2994359077 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3c84122c50 0x7f3c841230f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:58.973 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.967+0000 7f3c83fff6c0 1 -- 10.20.193.27:0/2994359077 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3c84115410 msgr2=0x7f3c8411d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:58.973 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.967+0000 7f3c83fff6c0 1 --2- 10.20.193.27:0/2994359077 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3c84115410 0x7f3c8411d890 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:58.973 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.967+0000 7f3c83fff6c0 1 -- 10.20.193.27:0/2994359077 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c70009590 con 0x7f3c8410f200 2026-02-13T02:25:58.974 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.967+0000 7f3c837fe6c0 1 --2- 10.20.193.27:0/2994359077 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3c84115410 0x7f3c8411d890 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:25:58.974 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.967+0000 7f3c88e166c0 1 --2- 10.20.193.27:0/2994359077 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3c84122c50 0x7f3c841230f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:25:58.974 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.967+0000 7f3c83fff6c0 1 --2- 10.20.193.27:0/2994359077 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3c8410f200 0x7f3c8411d350 secure :-1 s=READY pgs=457 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3c7002f230 tx=0x7f3c7002fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:58.974 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.967+0000 7f3c817fa6c0 1 -- 10.20.193.27:0/2994359077 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3c70002ba0 con 0x7f3c8410f200 2026-02-13T02:25:58.974 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.967+0000 7f3c8a8a46c0 1 -- 10.20.193.27:0/2994359077 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c84163de0 con 0x7f3c8410f200 2026-02-13T02:25:58.974 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.967+0000 7f3c817fa6c0 1 -- 10.20.193.27:0/2994359077 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3c70002d00 con 0x7f3c8410f200 2026-02-13T02:25:58.974 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.967+0000 7f3c817fa6c0 1 -- 10.20.193.27:0/2994359077 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3c700372f0 con 0x7f3c8410f200 2026-02-13T02:25:58.974 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.967+0000 7f3c8a8a46c0 1 -- 10.20.193.27:0/2994359077 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c84164210 con 0x7f3c8410f200 2026-02-13T02:25:58.974 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.968+0000 7f3c8a8a46c0 1 -- 10.20.193.27:0/2994359077 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c841104d0 con 0x7f3c8410f200 2026-02-13T02:25:58.975 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.969+0000 7f3c817fa6c0 1 -- 10.20.193.27:0/2994359077 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f3c700078d0 con 0x7f3c8410f200 2026-02-13T02:25:58.975 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.970+0000 7f3c817fa6c0 1 --2- 10.20.193.27:0/2994359077 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3c580b5600 0x7f3c580d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:25:58.975 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.970+0000 7f3c837fe6c0 1 --2- 10.20.193.27:0/2994359077 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3c580b5600 0x7f3c580d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:25:58.976 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.970+0000 7f3c817fa6c0 1 -- 10.20.193.27:0/2994359077 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f3c70107be0 con 0x7f3c8410f200 2026-02-13T02:25:58.976 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.970+0000 7f3c817fa6c0 1 -- 10.20.193.27:0/2994359077 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f3c700ca6a0 con 0x7f3c8410f200 2026-02-13T02:25:58.976 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:58.970+0000 7f3c837fe6c0 1 --2- 10.20.193.27:0/2994359077 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3c580b5600 0x7f3c580d5ab0 secure :-1 s=READY pgs=292 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3c84123ce0 tx=0x7f3c74005f10 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:25:59.085 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:59.083+0000 7f3c8a8a46c0 1 -- 10.20.193.27:0/2994359077 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3c8410f5d0 con 0x7f3c8410f200 2026-02-13T02:25:59.086 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:59.083+0000 7f3c817fa6c0 1 -- 10.20.193.27:0/2994359077 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f3c700cd850 con 0x7f3c8410f200 2026-02-13T02:25:59.086 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:25:59.086 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:25:59.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:59.084+0000 7f3c8a8a46c0 1 -- 10.20.193.27:0/2994359077 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3c580b5600 msgr2=0x7f3c580d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:59.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:59.084+0000 7f3c8a8a46c0 1 --2- 10.20.193.27:0/2994359077 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3c580b5600 0x7f3c580d5ab0 secure :-1 s=READY pgs=292 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3c84123ce0 tx=0x7f3c74005f10 comp rx=0 tx=0).stop 2026-02-13T02:25:59.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:59.084+0000 7f3c8a8a46c0 1 -- 10.20.193.27:0/2994359077 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3c8410f200 msgr2=0x7f3c8411d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:25:59.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:59.084+0000 7f3c8a8a46c0 1 --2- 10.20.193.27:0/2994359077 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3c8410f200 0x7f3c8411d350 secure :-1 s=READY pgs=457 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3c7002f230 tx=0x7f3c7002fd40 comp rx=0 tx=0).stop 2026-02-13T02:25:59.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:59.084+0000 7f3c8a8a46c0 1 -- 10.20.193.27:0/2994359077 shutdown_connections 2026-02-13T02:25:59.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:59.084+0000 7f3c8a8a46c0 1 --2- 10.20.193.27:0/2994359077 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3c580b5600 0x7f3c580d5ab0 unknown :-1 s=CLOSED pgs=292 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:59.088 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:59.084+0000 7f3c8a8a46c0 1 --2- 10.20.193.27:0/2994359077 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3c84122c50 0x7f3c841230f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:59.088 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:59.084+0000 7f3c8a8a46c0 1 --2- 10.20.193.27:0/2994359077 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3c84115410 0x7f3c8411d890 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:59.088 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:59.084+0000 7f3c8a8a46c0 1 --2- 10.20.193.27:0/2994359077 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3c8410f200 0x7f3c8411d350 unknown :-1 s=CLOSED pgs=457 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:25:59.088 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:59.084+0000 7f3c8a8a46c0 1 -- 10.20.193.27:0/2994359077 >> 10.20.193.27:0/2994359077 conn(0x7f3c840ecb00 msgr2=0x7f3c8410cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:25:59.088 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:59.084+0000 7f3c8a8a46c0 1 -- 10.20.193.27:0/2994359077 shutdown_connections 2026-02-13T02:25:59.088 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:25:59.084+0000 7f3c8a8a46c0 1 -- 10.20.193.27:0/2994359077 wait complete. 2026-02-13T02:25:59.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:59 trial027 ceph-mon[24567]: pgmap v300: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:59.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:25:59 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2994359077' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:59.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:59 trial028 ceph-mon[26701]: pgmap v300: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:59.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:25:59 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2994359077' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:25:59.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:59 trial107 ceph-mon[27422]: pgmap v300: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:25:59.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:25:59 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2994359077' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:01.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:01 trial027 ceph-mon[24567]: pgmap v301: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:01.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:01 trial028 ceph-mon[26701]: pgmap v301: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:01.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:01 trial107 ceph-mon[27422]: pgmap v301: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:02.127 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:26:02.247 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:26:02.364 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.360+0000 7f3a53c2c6c0 1 -- 10.20.193.27:0/970931377 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3a4c10f230 msgr2=0x7f3a4c10f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:02.364 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.360+0000 7f3a53c2c6c0 1 --2- 10.20.193.27:0/970931377 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3a4c10f230 0x7f3a4c10f600 secure :-1 s=READY pgs=458 gs=1 cs=0 l=1 c_cookie=11546009215803524062 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3a3c0099a0 tx=0x7f3a3c02ed20 comp rx=0 tx=0).stop 2026-02-13T02:26:02.364 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.360+0000 7f3a53c2c6c0 1 -- 10.20.193.27:0/970931377 shutdown_connections 2026-02-13T02:26:02.364 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.360+0000 7f3a53c2c6c0 1 --2- 10.20.193.27:0/970931377 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3a4c115440 0x7f3a4c175de0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:02.365 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.360+0000 7f3a53c2c6c0 1 --2- 10.20.193.27:0/970931377 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3a4c10fc20 0x7f3a4c114c60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:02.365 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.360+0000 7f3a53c2c6c0 1 --2- 10.20.193.27:0/970931377 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3a4c10f230 0x7f3a4c10f600 unknown :-1 s=CLOSED pgs=458 gs=1 cs=0 l=1 c_cookie=11546009215803524062 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:02.365 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.360+0000 7f3a53c2c6c0 1 -- 10.20.193.27:0/970931377 >> 10.20.193.27:0/970931377 conn(0x7f3a4c0ecb30 msgr2=0x7f3a4c10cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:02.365 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.360+0000 7f3a53c2c6c0 1 -- 10.20.193.27:0/970931377 shutdown_connections 2026-02-13T02:26:02.365 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.360+0000 7f3a53c2c6c0 1 -- 10.20.193.27:0/970931377 wait complete. 2026-02-13T02:26:02.365 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.360+0000 7f3a53c2c6c0 1 Processor -- start 2026-02-13T02:26:02.365 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.360+0000 7f3a53c2c6c0 1 -- start start 2026-02-13T02:26:02.365 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.360+0000 7f3a53c2c6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3a4c10f230 0x7f3a4c11d3b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:02.365 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.360+0000 7f3a53c2c6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3a4c115440 0x7f3a4c11d8f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:02.365 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.360+0000 7f3a53c2c6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3a4c122cb0 0x7f3a4c123150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:02.365 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.360+0000 7f3a53c2c6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c1236c0 con 0x7f3a4c115440 2026-02-13T02:26:02.366 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.360+0000 7f3a53c2c6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c123830 con 0x7f3a4c10f230 2026-02-13T02:26:02.366 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.360+0000 7f3a53c2c6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c1239f0 con 0x7f3a4c122cb0 2026-02-13T02:26:02.366 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.360+0000 7f3a511a06c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3a4c115440 0x7f3a4c11d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:02.366 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.360+0000 7f3a511a06c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3a4c115440 0x7f3a4c11d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:46040/0 (socket says 10.20.193.27:46040) 2026-02-13T02:26:02.366 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.360+0000 7f3a511a06c0 1 -- 10.20.193.27:0/3084090675 learned_addr learned my addr 10.20.193.27:0/3084090675 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:26:02.366 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.360+0000 7f3a519a16c0 1 --2- 10.20.193.27:0/3084090675 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3a4c10f230 0x7f3a4c11d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:02.366 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.360+0000 7f3a521a26c0 1 --2- 10.20.193.27:0/3084090675 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3a4c122cb0 0x7f3a4c123150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:02.366 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.360+0000 7f3a511a06c0 1 -- 10.20.193.27:0/3084090675 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3a4c122cb0 msgr2=0x7f3a4c123150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:02.366 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.360+0000 7f3a511a06c0 1 --2- 10.20.193.27:0/3084090675 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3a4c122cb0 0x7f3a4c123150 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:02.366 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.360+0000 7f3a511a06c0 1 -- 10.20.193.27:0/3084090675 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3a4c10f230 msgr2=0x7f3a4c11d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:02.366 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.360+0000 7f3a511a06c0 1 --2- 10.20.193.27:0/3084090675 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3a4c10f230 0x7f3a4c11d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:02.366 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.361+0000 7f3a511a06c0 1 -- 10.20.193.27:0/3084090675 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a3c009590 con 0x7f3a4c115440 2026-02-13T02:26:02.367 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.361+0000 7f3a511a06c0 1 --2- 10.20.193.27:0/3084090675 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3a4c115440 0x7f3a4c11d8f0 secure :-1 s=READY pgs=459 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3a40005dc0 tx=0x7f3a4000bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:02.367 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.361+0000 7f3a3affd6c0 1 -- 10.20.193.27:0/3084090675 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3a40004250 con 0x7f3a4c115440 2026-02-13T02:26:02.367 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.361+0000 7f3a53c2c6c0 1 -- 10.20.193.27:0/3084090675 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a4c163ea0 con 0x7f3a4c115440 2026-02-13T02:26:02.367 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.361+0000 7f3a3affd6c0 1 -- 10.20.193.27:0/3084090675 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3a40009070 con 0x7f3a4c115440 2026-02-13T02:26:02.367 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.361+0000 7f3a3affd6c0 1 -- 10.20.193.27:0/3084090675 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3a4000cd50 con 0x7f3a4c115440 2026-02-13T02:26:02.367 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.361+0000 7f3a53c2c6c0 1 -- 10.20.193.27:0/3084090675 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a4c164380 con 0x7f3a4c115440 2026-02-13T02:26:02.367 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.361+0000 7f3a53c2c6c0 1 -- 10.20.193.27:0/3084090675 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a4c110500 con 0x7f3a4c115440 2026-02-13T02:26:02.368 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.363+0000 7f3a3affd6c0 1 -- 10.20.193.27:0/3084090675 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f3a400028c0 con 0x7f3a4c115440 2026-02-13T02:26:02.368 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.364+0000 7f3a3affd6c0 1 --2- 10.20.193.27:0/3084090675 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3a280b5600 0x7f3a280d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:02.368 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.364+0000 7f3a519a16c0 1 --2- 10.20.193.27:0/3084090675 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3a280b5600 0x7f3a280d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:02.368 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.364+0000 7f3a3affd6c0 1 -- 10.20.193.27:0/3084090675 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f3a400e3a90 con 0x7f3a4c115440 2026-02-13T02:26:02.368 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.364+0000 7f3a519a16c0 1 --2- 10.20.193.27:0/3084090675 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3a280b5600 0x7f3a280d5ab0 secure :-1 s=READY pgs=293 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3a3c005b40 tx=0x7f3a3c0023a0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:02.368 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.364+0000 7f3a3affd6c0 1 -- 10.20.193.27:0/3084090675 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f3a400a6510 con 0x7f3a4c115440 2026-02-13T02:26:02.481 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.478+0000 7f3a53c2c6c0 1 -- 10.20.193.27:0/3084090675 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3a4c10f600 con 0x7f3a4c115440 2026-02-13T02:26:02.481 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.479+0000 7f3a3affd6c0 1 -- 10.20.193.27:0/3084090675 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f3a400a96c0 con 0x7f3a4c115440 2026-02-13T02:26:02.481 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:26:02.481 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:26:02.482 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.480+0000 7f3a53c2c6c0 1 -- 10.20.193.27:0/3084090675 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3a280b5600 msgr2=0x7f3a280d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:02.483 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.480+0000 7f3a53c2c6c0 1 --2- 10.20.193.27:0/3084090675 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3a280b5600 0x7f3a280d5ab0 secure :-1 s=READY pgs=293 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3a3c005b40 tx=0x7f3a3c0023a0 comp rx=0 tx=0).stop 2026-02-13T02:26:02.483 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.480+0000 7f3a53c2c6c0 1 -- 10.20.193.27:0/3084090675 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3a4c115440 msgr2=0x7f3a4c11d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:02.483 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.480+0000 7f3a53c2c6c0 1 --2- 10.20.193.27:0/3084090675 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3a4c115440 0x7f3a4c11d8f0 secure :-1 s=READY pgs=459 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3a40005dc0 tx=0x7f3a4000bbb0 comp rx=0 tx=0).stop 2026-02-13T02:26:02.483 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.480+0000 7f3a53c2c6c0 1 -- 10.20.193.27:0/3084090675 shutdown_connections 2026-02-13T02:26:02.483 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.480+0000 7f3a53c2c6c0 1 --2- 10.20.193.27:0/3084090675 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3a280b5600 0x7f3a280d5ab0 unknown :-1 s=CLOSED pgs=293 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:02.483 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.480+0000 7f3a53c2c6c0 1 --2- 10.20.193.27:0/3084090675 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3a4c122cb0 0x7f3a4c123150 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:02.483 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.480+0000 7f3a53c2c6c0 1 --2- 10.20.193.27:0/3084090675 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3a4c115440 0x7f3a4c11d8f0 unknown :-1 s=CLOSED pgs=459 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:02.483 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.480+0000 7f3a53c2c6c0 1 --2- 10.20.193.27:0/3084090675 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3a4c10f230 0x7f3a4c11d3b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:02.483 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.480+0000 7f3a53c2c6c0 1 -- 10.20.193.27:0/3084090675 >> 10.20.193.27:0/3084090675 conn(0x7f3a4c0ecb30 msgr2=0x7f3a4c10cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:02.484 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.480+0000 7f3a53c2c6c0 1 -- 10.20.193.27:0/3084090675 shutdown_connections 2026-02-13T02:26:02.484 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:02.480+0000 7f3a53c2c6c0 1 -- 10.20.193.27:0/3084090675 wait complete. 2026-02-13T02:26:03.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:03 trial027 ceph-mon[24567]: pgmap v302: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:03.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:03 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3084090675' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:03.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:03 trial028 ceph-mon[26701]: pgmap v302: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:03.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:03 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3084090675' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:03.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:03 trial107 ceph-mon[27422]: pgmap v302: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:03.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:03 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3084090675' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:05.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:05 trial027 ceph-mon[24567]: pgmap v303: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:26:05.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:05 trial028 ceph-mon[26701]: pgmap v303: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:26:05.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:05 trial107 ceph-mon[27422]: pgmap v303: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:26:05.522 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:26:05.641 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:26:05.759 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.756+0000 7fca61cf46c0 1 -- 10.20.193.27:0/2003951348 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fca5c115410 msgr2=0x7fca5c175db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:05.759 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.756+0000 7fca61cf46c0 1 --2- 10.20.193.27:0/2003951348 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fca5c115410 0x7fca5c175db0 secure :-1 s=READY pgs=460 gs=1 cs=0 l=1 c_cookie=12826680910073091509 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fca5000aa70 tx=0x7fca5002fde0 comp rx=0 tx=0).stop 2026-02-13T02:26:05.759 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.756+0000 7fca61cf46c0 1 -- 10.20.193.27:0/2003951348 shutdown_connections 2026-02-13T02:26:05.759 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.756+0000 7fca61cf46c0 1 --2- 10.20.193.27:0/2003951348 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fca5c115410 0x7fca5c175db0 unknown :-1 s=CLOSED pgs=460 gs=1 cs=0 l=1 c_cookie=12826680910073091509 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:05.759 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.756+0000 7fca61cf46c0 1 --2- 10.20.193.27:0/2003951348 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fca5c10fbf0 0x7fca5c114c30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:05.760 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.756+0000 7fca61cf46c0 1 --2- 10.20.193.27:0/2003951348 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fca5c10f200 0x7fca5c10f5d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:05.760 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.756+0000 7fca61cf46c0 1 -- 10.20.193.27:0/2003951348 >> 10.20.193.27:0/2003951348 conn(0x7fca5c0ecb00 msgr2=0x7fca5c10cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:05.760 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.756+0000 7fca61cf46c0 1 -- 10.20.193.27:0/2003951348 shutdown_connections 2026-02-13T02:26:05.760 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.756+0000 7fca61cf46c0 1 -- 10.20.193.27:0/2003951348 wait complete. 2026-02-13T02:26:05.760 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.756+0000 7fca61cf46c0 1 Processor -- start 2026-02-13T02:26:05.760 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.756+0000 7fca61cf46c0 1 -- start start 2026-02-13T02:26:05.760 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.756+0000 7fca61cf46c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fca5c10f200 0x7fca5c11d250 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:05.760 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.756+0000 7fca61cf46c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fca5c115410 0x7fca5c11d790 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:05.760 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.756+0000 7fca61cf46c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fca5c122b50 0x7fca5c122ff0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:05.760 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.756+0000 7fca61cf46c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c123560 con 0x7fca5c10f200 2026-02-13T02:26:05.760 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.756+0000 7fca61cf46c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c1236d0 con 0x7fca5c115410 2026-02-13T02:26:05.760 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.756+0000 7fca61cf46c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c123890 con 0x7fca5c122b50 2026-02-13T02:26:05.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.756+0000 7fca5b7fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fca5c10f200 0x7fca5c11d250 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:05.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.756+0000 7fca5b7fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fca5c10f200 0x7fca5c11d250 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:46064/0 (socket says 10.20.193.27:46064) 2026-02-13T02:26:05.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.756+0000 7fca5b7fe6c0 1 -- 10.20.193.27:0/3654792089 learned_addr learned my addr 10.20.193.27:0/3654792089 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:26:05.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.756+0000 7fca5affd6c0 1 --2- 10.20.193.27:0/3654792089 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fca5c115410 0x7fca5c11d790 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:05.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.756+0000 7fca5b7fe6c0 1 -- 10.20.193.27:0/3654792089 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fca5c122b50 msgr2=0x7fca5c122ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:05.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.756+0000 7fca5b7fe6c0 1 --2- 10.20.193.27:0/3654792089 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fca5c122b50 0x7fca5c122ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:05.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.756+0000 7fca5b7fe6c0 1 -- 10.20.193.27:0/3654792089 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fca5c115410 msgr2=0x7fca5c11d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:05.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.756+0000 7fca5b7fe6c0 1 --2- 10.20.193.27:0/3654792089 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fca5c115410 0x7fca5c11d790 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:05.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.756+0000 7fca5b7fe6c0 1 -- 10.20.193.27:0/3654792089 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca5000a660 con 0x7fca5c10f200 2026-02-13T02:26:05.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.757+0000 7fca5b7fe6c0 1 --2- 10.20.193.27:0/3654792089 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fca5c10f200 0x7fca5c11d250 secure :-1 s=READY pgs=461 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fca48009490 tx=0x7fca4800bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:05.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.757+0000 7fca58ff96c0 1 -- 10.20.193.27:0/3654792089 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fca48004a80 con 0x7fca5c10f200 2026-02-13T02:26:05.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.757+0000 7fca58ff96c0 1 -- 10.20.193.27:0/3654792089 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fca48009030 con 0x7fca5c10f200 2026-02-13T02:26:05.762 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.757+0000 7fca61cf46c0 1 -- 10.20.193.27:0/3654792089 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca5c163d40 con 0x7fca5c10f200 2026-02-13T02:26:05.762 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.757+0000 7fca58ff96c0 1 -- 10.20.193.27:0/3654792089 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fca4800ce30 con 0x7fca5c10f200 2026-02-13T02:26:05.762 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.757+0000 7fca61cf46c0 1 -- 10.20.193.27:0/3654792089 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca5c1641d0 con 0x7fca5c10f200 2026-02-13T02:26:05.762 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.757+0000 7fca61cf46c0 1 -- 10.20.193.27:0/3654792089 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca5c1104d0 con 0x7fca5c10f200 2026-02-13T02:26:05.763 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.758+0000 7fca58ff96c0 1 -- 10.20.193.27:0/3654792089 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7fca48004060 con 0x7fca5c10f200 2026-02-13T02:26:05.763 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.759+0000 7fca58ff96c0 1 --2- 10.20.193.27:0/3654792089 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fca300b5600 0x7fca300d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:05.763 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.759+0000 7fca58ff96c0 1 -- 10.20.193.27:0/3654792089 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7fca480e3b20 con 0x7fca5c10f200 2026-02-13T02:26:05.763 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.759+0000 7fca5affd6c0 1 --2- 10.20.193.27:0/3654792089 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fca300b5600 0x7fca300d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:05.763 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.759+0000 7fca5affd6c0 1 --2- 10.20.193.27:0/3654792089 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fca300b5600 0x7fca300d5ab0 secure :-1 s=READY pgs=294 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fca5c123be0 tx=0x7fca4c009260 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:05.763 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.759+0000 7fca58ff96c0 1 -- 10.20.193.27:0/3654792089 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fca480a65e0 con 0x7fca5c10f200 2026-02-13T02:26:05.877 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.876+0000 7fca61cf46c0 1 -- 10.20.193.27:0/3654792089 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fca5c10f5d0 con 0x7fca5c10f200 2026-02-13T02:26:05.877 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.876+0000 7fca58ff96c0 1 -- 10.20.193.27:0/3654792089 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7fca480a9790 con 0x7fca5c10f200 2026-02-13T02:26:05.878 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:26:05.878 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:26:05.879 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.878+0000 7fca61cf46c0 1 -- 10.20.193.27:0/3654792089 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fca300b5600 msgr2=0x7fca300d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:05.880 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.878+0000 7fca61cf46c0 1 --2- 10.20.193.27:0/3654792089 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fca300b5600 0x7fca300d5ab0 secure :-1 s=READY pgs=294 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fca5c123be0 tx=0x7fca4c009260 comp rx=0 tx=0).stop 2026-02-13T02:26:05.880 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.878+0000 7fca61cf46c0 1 -- 10.20.193.27:0/3654792089 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fca5c10f200 msgr2=0x7fca5c11d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:05.880 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.878+0000 7fca61cf46c0 1 --2- 10.20.193.27:0/3654792089 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fca5c10f200 0x7fca5c11d250 secure :-1 s=READY pgs=461 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fca48009490 tx=0x7fca4800bca0 comp rx=0 tx=0).stop 2026-02-13T02:26:05.880 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.878+0000 7fca61cf46c0 1 -- 10.20.193.27:0/3654792089 shutdown_connections 2026-02-13T02:26:05.880 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.878+0000 7fca61cf46c0 1 --2- 10.20.193.27:0/3654792089 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fca300b5600 0x7fca300d5ab0 unknown :-1 s=CLOSED pgs=294 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:05.880 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.878+0000 7fca61cf46c0 1 --2- 10.20.193.27:0/3654792089 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fca5c122b50 0x7fca5c122ff0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:05.880 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.878+0000 7fca61cf46c0 1 --2- 10.20.193.27:0/3654792089 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fca5c115410 0x7fca5c11d790 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:05.880 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.878+0000 7fca61cf46c0 1 --2- 10.20.193.27:0/3654792089 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fca5c10f200 0x7fca5c11d250 unknown :-1 s=CLOSED pgs=461 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:05.880 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.878+0000 7fca61cf46c0 1 -- 10.20.193.27:0/3654792089 >> 10.20.193.27:0/3654792089 conn(0x7fca5c0ecb00 msgr2=0x7fca5c10cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:05.880 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.878+0000 7fca61cf46c0 1 -- 10.20.193.27:0/3654792089 shutdown_connections 2026-02-13T02:26:05.881 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:05.878+0000 7fca61cf46c0 1 -- 10.20.193.27:0/3654792089 wait complete. 2026-02-13T02:26:06.158 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:06 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3654792089' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:06.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:06 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3654792089' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:06.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:06 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3654792089' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:07.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:07 trial027 ceph-mon[24567]: pgmap v304: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:07.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:07 trial028 ceph-mon[26701]: pgmap v304: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:07.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:07 trial107 ceph-mon[27422]: pgmap v304: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:08.923 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:26:09.044 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:26:09.174 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.170+0000 7fdc8c1bf6c0 1 -- 10.20.193.27:0/2300102960 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fdc8410f230 msgr2=0x7fdc8410f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:09.174 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.170+0000 7fdc8c1bf6c0 1 --2- 10.20.193.27:0/2300102960 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fdc8410f230 0x7fdc8410f600 secure :-1 s=READY pgs=462 gs=1 cs=0 l=1 c_cookie=6186943206938277525 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdc740099a0 tx=0x7fdc7402ed20 comp rx=0 tx=0).stop 2026-02-13T02:26:09.174 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.170+0000 7fdc8c1bf6c0 1 -- 10.20.193.27:0/2300102960 shutdown_connections 2026-02-13T02:26:09.174 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.170+0000 7fdc8c1bf6c0 1 --2- 10.20.193.27:0/2300102960 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdc84115440 0x7fdc84175de0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:09.174 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.170+0000 7fdc8c1bf6c0 1 --2- 10.20.193.27:0/2300102960 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fdc8410fc20 0x7fdc84114c60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:09.174 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.170+0000 7fdc8c1bf6c0 1 --2- 10.20.193.27:0/2300102960 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fdc8410f230 0x7fdc8410f600 unknown :-1 s=CLOSED pgs=462 gs=1 cs=0 l=1 c_cookie=6186943206938277525 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:09.174 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.170+0000 7fdc8c1bf6c0 1 -- 10.20.193.27:0/2300102960 >> 10.20.193.27:0/2300102960 conn(0x7fdc840ecb30 msgr2=0x7fdc8410cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:09.174 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.170+0000 7fdc8c1bf6c0 1 -- 10.20.193.27:0/2300102960 shutdown_connections 2026-02-13T02:26:09.175 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.170+0000 7fdc8c1bf6c0 1 -- 10.20.193.27:0/2300102960 wait complete. 2026-02-13T02:26:09.175 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.171+0000 7fdc8c1bf6c0 1 Processor -- start 2026-02-13T02:26:09.175 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.171+0000 7fdc8c1bf6c0 1 -- start start 2026-02-13T02:26:09.175 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.171+0000 7fdc8c1bf6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fdc8410f230 0x7fdc8411d280 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:09.175 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.171+0000 7fdc8c1bf6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fdc84115440 0x7fdc8411d7c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:09.175 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.171+0000 7fdc8c1bf6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdc84122b80 0x7fdc84123020 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:09.175 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.171+0000 7fdc8c1bf6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdc84123590 con 0x7fdc8410f230 2026-02-13T02:26:09.175 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.171+0000 7fdc8c1bf6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdc84123700 con 0x7fdc84122b80 2026-02-13T02:26:09.175 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.171+0000 7fdc8c1bf6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdc841238c0 con 0x7fdc84115440 2026-02-13T02:26:09.175 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.171+0000 7fdc89f306c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fdc8410f230 0x7fdc8411d280 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:09.175 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.171+0000 7fdc89f306c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fdc8410f230 0x7fdc8411d280 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:36472/0 (socket says 10.20.193.27:36472) 2026-02-13T02:26:09.175 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.171+0000 7fdc89f306c0 1 -- 10.20.193.27:0/1848749520 learned_addr learned my addr 10.20.193.27:0/1848749520 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:26:09.176 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.171+0000 7fdc8972f6c0 1 --2- 10.20.193.27:0/1848749520 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fdc84115440 0x7fdc8411d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:09.176 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.171+0000 7fdc8a7316c0 1 --2- 10.20.193.27:0/1848749520 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdc84122b80 0x7fdc84123020 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:09.176 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.171+0000 7fdc89f306c0 1 -- 10.20.193.27:0/1848749520 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fdc84115440 msgr2=0x7fdc8411d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:09.176 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.171+0000 7fdc89f306c0 1 --2- 10.20.193.27:0/1848749520 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fdc84115440 0x7fdc8411d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:09.176 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.171+0000 7fdc89f306c0 1 -- 10.20.193.27:0/1848749520 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdc84122b80 msgr2=0x7fdc84123020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:09.176 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.171+0000 7fdc89f306c0 1 --2- 10.20.193.27:0/1848749520 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdc84122b80 0x7fdc84123020 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:09.176 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.171+0000 7fdc89f306c0 1 -- 10.20.193.27:0/1848749520 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc74009590 con 0x7fdc8410f230 2026-02-13T02:26:09.176 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.171+0000 7fdc8a7316c0 1 --2- 10.20.193.27:0/1848749520 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdc84122b80 0x7fdc84123020 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:26:09.176 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.171+0000 7fdc89f306c0 1 --2- 10.20.193.27:0/1848749520 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fdc8410f230 0x7fdc8411d280 secure :-1 s=READY pgs=463 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdc7402f230 tx=0x7fdc7402fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:09.176 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.171+0000 7fdc72ffd6c0 1 -- 10.20.193.27:0/1848749520 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdc74002ba0 con 0x7fdc8410f230 2026-02-13T02:26:09.176 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.171+0000 7fdc8c1bf6c0 1 -- 10.20.193.27:0/1848749520 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc84163d10 con 0x7fdc8410f230 2026-02-13T02:26:09.176 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.171+0000 7fdc72ffd6c0 1 -- 10.20.193.27:0/1848749520 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fdc74007680 con 0x7fdc8410f230 2026-02-13T02:26:09.177 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.171+0000 7fdc72ffd6c0 1 -- 10.20.193.27:0/1848749520 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdc74037480 con 0x7fdc8410f230 2026-02-13T02:26:09.177 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.171+0000 7fdc8c1bf6c0 1 -- 10.20.193.27:0/1848749520 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc84164190 con 0x7fdc8410f230 2026-02-13T02:26:09.177 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.172+0000 7fdc8c1bf6c0 1 -- 10.20.193.27:0/1848749520 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc84110500 con 0x7fdc8410f230 2026-02-13T02:26:09.178 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.173+0000 7fdc72ffd6c0 1 -- 10.20.193.27:0/1848749520 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7fdc74002d20 con 0x7fdc8410f230 2026-02-13T02:26:09.178 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.173+0000 7fdc72ffd6c0 1 --2- 10.20.193.27:0/1848749520 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fdc600b5600 0x7fdc600d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:09.178 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.174+0000 7fdc8972f6c0 1 --2- 10.20.193.27:0/1848749520 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fdc600b5600 0x7fdc600d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:09.178 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.174+0000 7fdc72ffd6c0 1 -- 10.20.193.27:0/1848749520 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7fdc74107d60 con 0x7fdc8410f230 2026-02-13T02:26:09.178 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.174+0000 7fdc72ffd6c0 1 -- 10.20.193.27:0/1848749520 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fdc740ce280 con 0x7fdc8410f230 2026-02-13T02:26:09.178 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.174+0000 7fdc8972f6c0 1 --2- 10.20.193.27:0/1848749520 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fdc600b5600 0x7fdc600d5ab0 secure :-1 s=READY pgs=295 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdc78005980 tx=0x7fdc78005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:09.292 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.291+0000 7fdc8c1bf6c0 1 -- 10.20.193.27:0/1848749520 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fdc8410f600 con 0x7fdc8410f230 2026-02-13T02:26:09.293 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:09 trial027 ceph-mon[24567]: pgmap v305: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:26:09.293 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.291+0000 7fdc72ffd6c0 1 -- 10.20.193.27:0/1848749520 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7fdc740cd9d0 con 0x7fdc8410f230 2026-02-13T02:26:09.294 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:26:09.294 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:26:09.294 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.293+0000 7fdc8c1bf6c0 1 -- 10.20.193.27:0/1848749520 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fdc600b5600 msgr2=0x7fdc600d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:09.295 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.293+0000 7fdc8c1bf6c0 1 --2- 10.20.193.27:0/1848749520 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fdc600b5600 0x7fdc600d5ab0 secure :-1 s=READY pgs=295 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdc78005980 tx=0x7fdc78005910 comp rx=0 tx=0).stop 2026-02-13T02:26:09.295 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.293+0000 7fdc8c1bf6c0 1 -- 10.20.193.27:0/1848749520 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fdc8410f230 msgr2=0x7fdc8411d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:09.295 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.293+0000 7fdc8c1bf6c0 1 --2- 10.20.193.27:0/1848749520 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fdc8410f230 0x7fdc8411d280 secure :-1 s=READY pgs=463 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdc7402f230 tx=0x7fdc7402fd40 comp rx=0 tx=0).stop 2026-02-13T02:26:09.295 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.293+0000 7fdc8c1bf6c0 1 -- 10.20.193.27:0/1848749520 shutdown_connections 2026-02-13T02:26:09.295 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.293+0000 7fdc8c1bf6c0 1 --2- 10.20.193.27:0/1848749520 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fdc600b5600 0x7fdc600d5ab0 unknown :-1 s=CLOSED pgs=295 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:09.295 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.293+0000 7fdc8c1bf6c0 1 --2- 10.20.193.27:0/1848749520 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdc84122b80 0x7fdc84123020 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:09.295 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.293+0000 7fdc8c1bf6c0 1 --2- 10.20.193.27:0/1848749520 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fdc84115440 0x7fdc8411d7c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:09.295 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.293+0000 7fdc8c1bf6c0 1 --2- 10.20.193.27:0/1848749520 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fdc8410f230 0x7fdc8411d280 unknown :-1 s=CLOSED pgs=463 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:09.295 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.293+0000 7fdc8c1bf6c0 1 -- 10.20.193.27:0/1848749520 >> 10.20.193.27:0/1848749520 conn(0x7fdc840ecb30 msgr2=0x7fdc8410cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:09.295 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.293+0000 7fdc8c1bf6c0 1 -- 10.20.193.27:0/1848749520 shutdown_connections 2026-02-13T02:26:09.295 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:09.293+0000 7fdc8c1bf6c0 1 -- 10.20.193.27:0/1848749520 wait complete. 2026-02-13T02:26:09.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:09 trial028 ceph-mon[26701]: pgmap v305: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:26:09.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:09 trial107 ceph-mon[27422]: pgmap v305: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:26:10.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:10 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1848749520' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:10.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:10 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1848749520' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:10.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:10 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1848749520' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:11.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:11 trial027 ceph-mon[24567]: pgmap v306: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:11.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:11 trial028 ceph-mon[26701]: pgmap v306: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:11.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:11 trial107 ceph-mon[27422]: pgmap v306: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:12.337 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:26:12.457 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:26:12.579 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.576+0000 7f0b2e3246c0 1 -- 10.20.193.27:0/539631717 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0b2810fc20 msgr2=0x7f0b28114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:12.579 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.576+0000 7f0b2e3246c0 1 --2- 10.20.193.27:0/539631717 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0b2810fc20 0x7f0b28114c60 secure :-1 s=READY pgs=464 gs=1 cs=0 l=1 c_cookie=8425690209140710488 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0b180099a0 tx=0x7f0b1802ed00 comp rx=0 tx=0).stop 2026-02-13T02:26:12.579 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.576+0000 7f0b2e3246c0 1 -- 10.20.193.27:0/539631717 shutdown_connections 2026-02-13T02:26:12.579 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.576+0000 7f0b2e3246c0 1 --2- 10.20.193.27:0/539631717 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0b28115440 0x7f0b28175de0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:12.579 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.576+0000 7f0b2e3246c0 1 --2- 10.20.193.27:0/539631717 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0b2810fc20 0x7f0b28114c60 unknown :-1 s=CLOSED pgs=464 gs=1 cs=0 l=1 c_cookie=8425690209140710488 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:12.579 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.576+0000 7f0b2e3246c0 1 --2- 10.20.193.27:0/539631717 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f0b2810f230 0x7f0b2810f600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:12.579 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.576+0000 7f0b2e3246c0 1 -- 10.20.193.27:0/539631717 >> 10.20.193.27:0/539631717 conn(0x7f0b280ecb30 msgr2=0x7f0b2810cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:12.579 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.576+0000 7f0b2e3246c0 1 -- 10.20.193.27:0/539631717 shutdown_connections 2026-02-13T02:26:12.580 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.576+0000 7f0b2e3246c0 1 -- 10.20.193.27:0/539631717 wait complete. 2026-02-13T02:26:12.580 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.576+0000 7f0b2e3246c0 1 Processor -- start 2026-02-13T02:26:12.580 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.576+0000 7f0b2e3246c0 1 -- start start 2026-02-13T02:26:12.580 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.576+0000 7f0b2e3246c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0b2810f230 0x7f0b280f4b80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:12.580 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.576+0000 7f0b2e3246c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0b28115440 0x7f0b28175ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:12.580 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.576+0000 7f0b2e3246c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f0b280f51b0 0x7f0b280fa480 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:12.580 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.576+0000 7f0b2e3246c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0b280f5810 con 0x7f0b28115440 2026-02-13T02:26:12.580 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.576+0000 7f0b2e3246c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0b280fa9c0 con 0x7f0b2810f230 2026-02-13T02:26:12.580 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.576+0000 7f0b2e3246c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0b280fab80 con 0x7f0b280f51b0 2026-02-13T02:26:12.580 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.576+0000 7f0b277fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0b28115440 0x7f0b28175ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:12.580 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.576+0000 7f0b277fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0b28115440 0x7f0b28175ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:36482/0 (socket says 10.20.193.27:36482) 2026-02-13T02:26:12.580 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.576+0000 7f0b277fe6c0 1 -- 10.20.193.27:0/4035622893 learned_addr learned my addr 10.20.193.27:0/4035622893 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:26:12.581 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.576+0000 7f0b277fe6c0 1 -- 10.20.193.27:0/4035622893 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f0b280f51b0 msgr2=0x7f0b280fa480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:12.581 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.576+0000 7f0b277fe6c0 1 --2- 10.20.193.27:0/4035622893 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f0b280f51b0 0x7f0b280fa480 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:12.581 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.576+0000 7f0b277fe6c0 1 -- 10.20.193.27:0/4035622893 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0b2810f230 msgr2=0x7f0b280f4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:12.581 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.576+0000 7f0b277fe6c0 1 --2- 10.20.193.27:0/4035622893 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0b2810f230 0x7f0b280f4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:12.581 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.576+0000 7f0b277fe6c0 1 -- 10.20.193.27:0/4035622893 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b18009590 con 0x7f0b28115440 2026-02-13T02:26:12.581 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.576+0000 7f0b277fe6c0 1 --2- 10.20.193.27:0/4035622893 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0b28115440 0x7f0b28175ce0 secure :-1 s=READY pgs=465 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0b180023c0 tx=0x7f0b1802fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:12.581 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.576+0000 7f0b257fa6c0 1 -- 10.20.193.27:0/4035622893 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0b18002ba0 con 0x7f0b28115440 2026-02-13T02:26:12.581 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.576+0000 7f0b2e3246c0 1 -- 10.20.193.27:0/4035622893 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b280fada0 con 0x7f0b28115440 2026-02-13T02:26:12.581 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.576+0000 7f0b257fa6c0 1 -- 10.20.193.27:0/4035622893 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f0b18007680 con 0x7f0b28115440 2026-02-13T02:26:12.581 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.577+0000 7f0b257fa6c0 1 -- 10.20.193.27:0/4035622893 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0b18031420 con 0x7f0b28115440 2026-02-13T02:26:12.581 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.577+0000 7f0b2e3246c0 1 -- 10.20.193.27:0/4035622893 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b280fb270 con 0x7f0b28115440 2026-02-13T02:26:12.581 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.577+0000 7f0b2e3246c0 1 -- 10.20.193.27:0/4035622893 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b28110500 con 0x7f0b28115440 2026-02-13T02:26:12.582 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.578+0000 7f0b257fa6c0 1 -- 10.20.193.27:0/4035622893 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f0b180071c0 con 0x7f0b28115440 2026-02-13T02:26:12.583 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.579+0000 7f0b257fa6c0 1 --2- 10.20.193.27:0/4035622893 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f0afc0b5600 0x7f0afc0d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:12.583 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.579+0000 7f0b27fff6c0 1 --2- 10.20.193.27:0/4035622893 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f0afc0b5600 0x7f0afc0d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:12.583 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.579+0000 7f0b257fa6c0 1 -- 10.20.193.27:0/4035622893 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f0b18107f00 con 0x7f0b28115440 2026-02-13T02:26:12.583 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.579+0000 7f0b27fff6c0 1 --2- 10.20.193.27:0/4035622893 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f0afc0b5600 0x7f0afc0d5ab0 secure :-1 s=READY pgs=296 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0b14004750 tx=0x7f0b14005d30 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:12.583 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.579+0000 7f0b257fa6c0 1 -- 10.20.193.27:0/4035622893 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f0b180ca980 con 0x7f0b28115440 2026-02-13T02:26:12.697 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.695+0000 7f0b2e3246c0 1 -- 10.20.193.27:0/4035622893 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0b280f5d00 con 0x7f0b28115440 2026-02-13T02:26:12.697 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.695+0000 7f0b257fa6c0 1 -- 10.20.193.27:0/4035622893 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f0b180cdb30 con 0x7f0b28115440 2026-02-13T02:26:12.697 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:26:12.697 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:26:12.698 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.696+0000 7f0b2e3246c0 1 -- 10.20.193.27:0/4035622893 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f0afc0b5600 msgr2=0x7f0afc0d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:12.698 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.696+0000 7f0b2e3246c0 1 --2- 10.20.193.27:0/4035622893 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f0afc0b5600 0x7f0afc0d5ab0 secure :-1 s=READY pgs=296 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0b14004750 tx=0x7f0b14005d30 comp rx=0 tx=0).stop 2026-02-13T02:26:12.699 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.696+0000 7f0b2e3246c0 1 -- 10.20.193.27:0/4035622893 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0b28115440 msgr2=0x7f0b28175ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:12.699 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.696+0000 7f0b2e3246c0 1 --2- 10.20.193.27:0/4035622893 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0b28115440 0x7f0b28175ce0 secure :-1 s=READY pgs=465 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0b180023c0 tx=0x7f0b1802fc60 comp rx=0 tx=0).stop 2026-02-13T02:26:12.699 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.697+0000 7f0b2e3246c0 1 -- 10.20.193.27:0/4035622893 shutdown_connections 2026-02-13T02:26:12.699 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.697+0000 7f0b2e3246c0 1 --2- 10.20.193.27:0/4035622893 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f0afc0b5600 0x7f0afc0d5ab0 unknown :-1 s=CLOSED pgs=296 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:12.699 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.697+0000 7f0b2e3246c0 1 --2- 10.20.193.27:0/4035622893 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f0b280f51b0 0x7f0b280fa480 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:12.699 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.697+0000 7f0b2e3246c0 1 --2- 10.20.193.27:0/4035622893 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0b28115440 0x7f0b28175ce0 unknown :-1 s=CLOSED pgs=465 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:12.699 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.697+0000 7f0b2e3246c0 1 --2- 10.20.193.27:0/4035622893 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0b2810f230 0x7f0b280f4b80 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:12.699 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.697+0000 7f0b2e3246c0 1 -- 10.20.193.27:0/4035622893 >> 10.20.193.27:0/4035622893 conn(0x7f0b280ecb30 msgr2=0x7f0b2810d850 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:12.699 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.697+0000 7f0b2e3246c0 1 -- 10.20.193.27:0/4035622893 shutdown_connections 2026-02-13T02:26:12.699 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:12.697+0000 7f0b2e3246c0 1 -- 10.20.193.27:0/4035622893 wait complete. 2026-02-13T02:26:13.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:13 trial027 ceph-mon[24567]: pgmap v307: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:13.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:13 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/4035622893' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:13.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:13 trial028 ceph-mon[26701]: pgmap v307: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:13.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:13 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/4035622893' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:13.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:13 trial107 ceph-mon[27422]: pgmap v307: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:13.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:13 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/4035622893' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:15.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:15 trial027 ceph-mon[24567]: pgmap v308: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:26:15.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:15 trial028 ceph-mon[26701]: pgmap v308: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:26:15.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:15 trial107 ceph-mon[27422]: pgmap v308: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:26:15.739 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:26:15.855 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:26:15.971 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.967+0000 7f4dca4ed6c0 1 -- 10.20.193.27:0/2519719014 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4dc410f200 msgr2=0x7f4dc410f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:15.971 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.967+0000 7f4dca4ed6c0 1 --2- 10.20.193.27:0/2519719014 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4dc410f200 0x7f4dc410f5d0 secure :-1 s=READY pgs=466 gs=1 cs=0 l=1 c_cookie=12865586896607403265 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4db00099a0 tx=0x7f4db002ed20 comp rx=0 tx=0).stop 2026-02-13T02:26:15.971 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.967+0000 7f4dca4ed6c0 1 -- 10.20.193.27:0/2519719014 shutdown_connections 2026-02-13T02:26:15.971 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.967+0000 7f4dca4ed6c0 1 --2- 10.20.193.27:0/2519719014 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f4dc4115410 0x7f4dc4175db0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:15.971 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.967+0000 7f4dca4ed6c0 1 --2- 10.20.193.27:0/2519719014 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4dc410fbf0 0x7f4dc4114c30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:15.971 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.967+0000 7f4dca4ed6c0 1 --2- 10.20.193.27:0/2519719014 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4dc410f200 0x7f4dc410f5d0 unknown :-1 s=CLOSED pgs=466 gs=1 cs=0 l=1 c_cookie=12865586896607403265 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:15.971 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.967+0000 7f4dca4ed6c0 1 -- 10.20.193.27:0/2519719014 >> 10.20.193.27:0/2519719014 conn(0x7f4dc40ecb00 msgr2=0x7f4dc410cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:15.972 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.967+0000 7f4dca4ed6c0 1 -- 10.20.193.27:0/2519719014 shutdown_connections 2026-02-13T02:26:15.972 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.967+0000 7f4dca4ed6c0 1 -- 10.20.193.27:0/2519719014 wait complete. 2026-02-13T02:26:15.972 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.968+0000 7f4dca4ed6c0 1 Processor -- start 2026-02-13T02:26:15.972 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.968+0000 7f4dca4ed6c0 1 -- start start 2026-02-13T02:26:15.972 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.968+0000 7f4dca4ed6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f4dc410f200 0x7f4dc411d260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:15.972 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.968+0000 7f4dca4ed6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4dc4115410 0x7f4dc411d7a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:15.972 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.968+0000 7f4dca4ed6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4dc4122b60 0x7f4dc4123000 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:15.972 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.968+0000 7f4dca4ed6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc4123570 con 0x7f4dc4122b60 2026-02-13T02:26:15.972 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.968+0000 7f4dca4ed6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc41236e0 con 0x7f4dc4115410 2026-02-13T02:26:15.972 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.968+0000 7f4dca4ed6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc41238a0 con 0x7f4dc410f200 2026-02-13T02:26:15.972 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.968+0000 7f4dc8a5f6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4dc4122b60 0x7f4dc4123000 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:15.972 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.968+0000 7f4dc8a5f6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4dc4122b60 0x7f4dc4123000 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:36514/0 (socket says 10.20.193.27:36514) 2026-02-13T02:26:15.973 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.968+0000 7f4dc8a5f6c0 1 -- 10.20.193.27:0/3990815360 learned_addr learned my addr 10.20.193.27:0/3990815360 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:26:15.973 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.968+0000 7f4dc37fe6c0 1 --2- 10.20.193.27:0/3990815360 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4dc4115410 0x7f4dc411d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:15.973 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.968+0000 7f4dc3fff6c0 1 --2- 10.20.193.27:0/3990815360 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f4dc410f200 0x7f4dc411d260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:15.973 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.968+0000 7f4dc8a5f6c0 1 -- 10.20.193.27:0/3990815360 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f4dc410f200 msgr2=0x7f4dc411d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:15.973 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.968+0000 7f4dc8a5f6c0 1 --2- 10.20.193.27:0/3990815360 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f4dc410f200 0x7f4dc411d260 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:15.973 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.968+0000 7f4dc8a5f6c0 1 -- 10.20.193.27:0/3990815360 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4dc4115410 msgr2=0x7f4dc411d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:15.973 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.968+0000 7f4dc8a5f6c0 1 --2- 10.20.193.27:0/3990815360 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4dc4115410 0x7f4dc411d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:15.973 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.968+0000 7f4dc8a5f6c0 1 -- 10.20.193.27:0/3990815360 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db0009590 con 0x7f4dc4122b60 2026-02-13T02:26:15.973 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.968+0000 7f4dc8a5f6c0 1 --2- 10.20.193.27:0/3990815360 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4dc4122b60 0x7f4dc4123000 secure :-1 s=READY pgs=467 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4db800c6c0 tx=0x7f4db800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:15.973 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.968+0000 7f4dc17fa6c0 1 -- 10.20.193.27:0/3990815360 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4db8003c40 con 0x7f4dc4122b60 2026-02-13T02:26:15.973 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.968+0000 7f4dc17fa6c0 1 -- 10.20.193.27:0/3990815360 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4db800a030 con 0x7f4dc4122b60 2026-02-13T02:26:15.974 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.968+0000 7f4dca4ed6c0 1 -- 10.20.193.27:0/3990815360 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dc4163d50 con 0x7f4dc4122b60 2026-02-13T02:26:15.974 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.968+0000 7f4dc17fa6c0 1 -- 10.20.193.27:0/3990815360 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4db800db70 con 0x7f4dc4122b60 2026-02-13T02:26:15.974 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.968+0000 7f4dca4ed6c0 1 -- 10.20.193.27:0/3990815360 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dc4164230 con 0x7f4dc4122b60 2026-02-13T02:26:15.974 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.969+0000 7f4dca4ed6c0 1 -- 10.20.193.27:0/3990815360 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dc41104d0 con 0x7f4dc4122b60 2026-02-13T02:26:15.975 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.970+0000 7f4dc17fa6c0 1 -- 10.20.193.27:0/3990815360 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f4db800dd20 con 0x7f4dc4122b60 2026-02-13T02:26:15.975 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.971+0000 7f4dc17fa6c0 1 --2- 10.20.193.27:0/3990815360 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f4d980b5680 0x7f4d980d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:15.975 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.971+0000 7f4dc17fa6c0 1 -- 10.20.193.27:0/3990815360 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f4db80e4930 con 0x7f4dc4122b60 2026-02-13T02:26:15.975 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.971+0000 7f4dc3fff6c0 1 --2- 10.20.193.27:0/3990815360 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f4d980b5680 0x7f4d980d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:15.975 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.971+0000 7f4dc17fa6c0 1 -- 10.20.193.27:0/3990815360 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f4db80aa380 con 0x7f4dc4122b60 2026-02-13T02:26:15.975 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:15.971+0000 7f4dc3fff6c0 1 --2- 10.20.193.27:0/3990815360 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f4d980b5680 0x7f4d980d5b30 secure :-1 s=READY pgs=297 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4db00023e0 tx=0x7f4db0005c60 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:16.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:16.086+0000 7f4dca4ed6c0 1 -- 10.20.193.27:0/3990815360 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4dc410f5d0 con 0x7f4dc4122b60 2026-02-13T02:26:16.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:16.086+0000 7f4dc17fa6c0 1 -- 10.20.193.27:0/3990815360 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f4db80aa560 con 0x7f4dc4122b60 2026-02-13T02:26:16.088 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:26:16.088 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:26:16.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:16.087+0000 7f4dca4ed6c0 1 -- 10.20.193.27:0/3990815360 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f4d980b5680 msgr2=0x7f4d980d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:16.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:16.087+0000 7f4dca4ed6c0 1 --2- 10.20.193.27:0/3990815360 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f4d980b5680 0x7f4d980d5b30 secure :-1 s=READY pgs=297 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4db00023e0 tx=0x7f4db0005c60 comp rx=0 tx=0).stop 2026-02-13T02:26:16.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:16.087+0000 7f4dca4ed6c0 1 -- 10.20.193.27:0/3990815360 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4dc4122b60 msgr2=0x7f4dc4123000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:16.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:16.087+0000 7f4dca4ed6c0 1 --2- 10.20.193.27:0/3990815360 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4dc4122b60 0x7f4dc4123000 secure :-1 s=READY pgs=467 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4db800c6c0 tx=0x7f4db800cb90 comp rx=0 tx=0).stop 2026-02-13T02:26:16.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:16.087+0000 7f4dca4ed6c0 1 -- 10.20.193.27:0/3990815360 shutdown_connections 2026-02-13T02:26:16.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:16.087+0000 7f4dca4ed6c0 1 --2- 10.20.193.27:0/3990815360 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f4d980b5680 0x7f4d980d5b30 unknown :-1 s=CLOSED pgs=297 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:16.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:16.087+0000 7f4dca4ed6c0 1 --2- 10.20.193.27:0/3990815360 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4dc4122b60 0x7f4dc4123000 unknown :-1 s=CLOSED pgs=467 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:16.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:16.087+0000 7f4dca4ed6c0 1 --2- 10.20.193.27:0/3990815360 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4dc4115410 0x7f4dc411d7a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:16.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:16.087+0000 7f4dca4ed6c0 1 --2- 10.20.193.27:0/3990815360 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f4dc410f200 0x7f4dc411d260 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:16.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:16.087+0000 7f4dca4ed6c0 1 -- 10.20.193.27:0/3990815360 >> 10.20.193.27:0/3990815360 conn(0x7f4dc40ecb00 msgr2=0x7f4dc410cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:16.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:16.087+0000 7f4dca4ed6c0 1 -- 10.20.193.27:0/3990815360 shutdown_connections 2026-02-13T02:26:16.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:16.087+0000 7f4dca4ed6c0 1 -- 10.20.193.27:0/3990815360 wait complete. 2026-02-13T02:26:16.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:16 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3990815360' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:16.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:16 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3990815360' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:16.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:16 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3990815360' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:17.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:17 trial027 ceph-mon[24567]: pgmap v309: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:17.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:17 trial028 ceph-mon[26701]: pgmap v309: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:17.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:17 trial107 ceph-mon[27422]: pgmap v309: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:19.128 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:26:19.248 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:26:19.362 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.357+0000 7f8d5e6556c0 1 -- 10.20.193.27:0/980591266 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8d58115410 msgr2=0x7f8d58175db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:19.362 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.357+0000 7f8d5e6556c0 1 --2- 10.20.193.27:0/980591266 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8d58115410 0x7f8d58175db0 secure :-1 s=READY pgs=468 gs=1 cs=0 l=1 c_cookie=6977168490021601484 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8d4c00aa70 tx=0x7f8d4c02fde0 comp rx=0 tx=0).stop 2026-02-13T02:26:19.362 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.357+0000 7f8d5e6556c0 1 -- 10.20.193.27:0/980591266 shutdown_connections 2026-02-13T02:26:19.362 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.357+0000 7f8d5e6556c0 1 --2- 10.20.193.27:0/980591266 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8d58115410 0x7f8d58175db0 unknown :-1 s=CLOSED pgs=468 gs=1 cs=0 l=1 c_cookie=6977168490021601484 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:19.362 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.357+0000 7f8d5e6556c0 1 --2- 10.20.193.27:0/980591266 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8d5810fbf0 0x7f8d58114c30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:19.362 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.357+0000 7f8d5e6556c0 1 --2- 10.20.193.27:0/980591266 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8d5810f200 0x7f8d5810f5d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:19.362 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.357+0000 7f8d5e6556c0 1 -- 10.20.193.27:0/980591266 >> 10.20.193.27:0/980591266 conn(0x7f8d580ecb00 msgr2=0x7f8d5810cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:19.362 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.357+0000 7f8d5e6556c0 1 -- 10.20.193.27:0/980591266 shutdown_connections 2026-02-13T02:26:19.362 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.357+0000 7f8d5e6556c0 1 -- 10.20.193.27:0/980591266 wait complete. 2026-02-13T02:26:19.363 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.357+0000 7f8d5e6556c0 1 Processor -- start 2026-02-13T02:26:19.363 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.358+0000 7f8d5e6556c0 1 -- start start 2026-02-13T02:26:19.363 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.358+0000 7f8d5e6556c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8d5810f200 0x7f8d5811d1d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:19.363 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.358+0000 7f8d5e6556c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8d58115410 0x7f8d5811d710 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:19.363 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.358+0000 7f8d5e6556c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8d58122ad0 0x7f8d58122f70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:19.363 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.358+0000 7f8d5e6556c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8d581234e0 con 0x7f8d58115410 2026-02-13T02:26:19.363 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.358+0000 7f8d5e6556c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8d58123650 con 0x7f8d58122ad0 2026-02-13T02:26:19.363 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.358+0000 7f8d5e6556c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8d58123810 con 0x7f8d5810f200 2026-02-13T02:26:19.363 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.358+0000 7f8d577fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8d58115410 0x7f8d5811d710 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:19.363 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.358+0000 7f8d577fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8d58115410 0x7f8d5811d710 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:35198/0 (socket says 10.20.193.27:35198) 2026-02-13T02:26:19.363 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.358+0000 7f8d577fe6c0 1 -- 10.20.193.27:0/3603816533 learned_addr learned my addr 10.20.193.27:0/3603816533 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:26:19.364 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.358+0000 7f8d57fff6c0 1 --2- 10.20.193.27:0/3603816533 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8d5810f200 0x7f8d5811d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:19.364 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.358+0000 7f8d577fe6c0 1 -- 10.20.193.27:0/3603816533 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8d5810f200 msgr2=0x7f8d5811d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:19.364 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.358+0000 7f8d577fe6c0 1 --2- 10.20.193.27:0/3603816533 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8d5810f200 0x7f8d5811d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:19.364 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.358+0000 7f8d5cbc76c0 1 --2- 10.20.193.27:0/3603816533 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8d58122ad0 0x7f8d58122f70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:19.364 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.358+0000 7f8d577fe6c0 1 -- 10.20.193.27:0/3603816533 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8d58122ad0 msgr2=0x7f8d58122f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:19.364 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.358+0000 7f8d577fe6c0 1 --2- 10.20.193.27:0/3603816533 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8d58122ad0 0x7f8d58122f70 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:19.364 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.358+0000 7f8d577fe6c0 1 -- 10.20.193.27:0/3603816533 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d4c00a660 con 0x7f8d58115410 2026-02-13T02:26:19.364 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.358+0000 7f8d5cbc76c0 1 --2- 10.20.193.27:0/3603816533 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8d58122ad0 0x7f8d58122f70 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:26:19.364 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.358+0000 7f8d577fe6c0 1 --2- 10.20.193.27:0/3603816533 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8d58115410 0x7f8d5811d710 secure :-1 s=READY pgs=469 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8d4800b800 tx=0x7f8d4800bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:19.364 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.358+0000 7f8d557fa6c0 1 -- 10.20.193.27:0/3603816533 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8d48004a80 con 0x7f8d58115410 2026-02-13T02:26:19.364 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.358+0000 7f8d5e6556c0 1 -- 10.20.193.27:0/3603816533 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d58163cc0 con 0x7f8d58115410 2026-02-13T02:26:19.364 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.358+0000 7f8d557fa6c0 1 -- 10.20.193.27:0/3603816533 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f8d48009070 con 0x7f8d58115410 2026-02-13T02:26:19.365 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.358+0000 7f8d557fa6c0 1 -- 10.20.193.27:0/3603816533 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8d4800cdd0 con 0x7f8d58115410 2026-02-13T02:26:19.365 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.358+0000 7f8d5e6556c0 1 -- 10.20.193.27:0/3603816533 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d58164150 con 0x7f8d58115410 2026-02-13T02:26:19.365 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.359+0000 7f8d5e6556c0 1 -- 10.20.193.27:0/3603816533 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d581104d0 con 0x7f8d58115410 2026-02-13T02:26:19.366 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.360+0000 7f8d557fa6c0 1 -- 10.20.193.27:0/3603816533 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f8d48004060 con 0x7f8d58115410 2026-02-13T02:26:19.366 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.361+0000 7f8d557fa6c0 1 --2- 10.20.193.27:0/3603816533 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8d2c0b5600 0x7f8d2c0d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:19.366 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.361+0000 7f8d57fff6c0 1 --2- 10.20.193.27:0/3603816533 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8d2c0b5600 0x7f8d2c0d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:19.366 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.361+0000 7f8d557fa6c0 1 -- 10.20.193.27:0/3603816533 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f8d480e3aa0 con 0x7f8d58115410 2026-02-13T02:26:19.366 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.361+0000 7f8d57fff6c0 1 --2- 10.20.193.27:0/3603816533 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8d2c0b5600 0x7f8d2c0d5ab0 secure :-1 s=READY pgs=298 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8d44004750 tx=0x7f8d44009260 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:19.366 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.361+0000 7f8d557fa6c0 1 -- 10.20.193.27:0/3603816533 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f8d480a9fc0 con 0x7f8d58115410 2026-02-13T02:26:19.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:19 trial027 ceph-mon[24567]: pgmap v310: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:26:19.477 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.475+0000 7f8d5e6556c0 1 -- 10.20.193.27:0/3603816533 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8d5810fbf0 con 0x7f8d58115410 2026-02-13T02:26:19.478 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.475+0000 7f8d557fa6c0 1 -- 10.20.193.27:0/3603816533 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f8d480a9710 con 0x7f8d58115410 2026-02-13T02:26:19.478 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:26:19.478 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:26:19.479 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.476+0000 7f8d5e6556c0 1 -- 10.20.193.27:0/3603816533 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8d2c0b5600 msgr2=0x7f8d2c0d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:19.479 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.476+0000 7f8d5e6556c0 1 --2- 10.20.193.27:0/3603816533 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8d2c0b5600 0x7f8d2c0d5ab0 secure :-1 s=READY pgs=298 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8d44004750 tx=0x7f8d44009260 comp rx=0 tx=0).stop 2026-02-13T02:26:19.479 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.476+0000 7f8d5e6556c0 1 -- 10.20.193.27:0/3603816533 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8d58115410 msgr2=0x7f8d5811d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:19.480 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.476+0000 7f8d5e6556c0 1 --2- 10.20.193.27:0/3603816533 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8d58115410 0x7f8d5811d710 secure :-1 s=READY pgs=469 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8d4800b800 tx=0x7f8d4800bcd0 comp rx=0 tx=0).stop 2026-02-13T02:26:19.480 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.476+0000 7f8d5e6556c0 1 -- 10.20.193.27:0/3603816533 shutdown_connections 2026-02-13T02:26:19.480 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.476+0000 7f8d5e6556c0 1 --2- 10.20.193.27:0/3603816533 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f8d2c0b5600 0x7f8d2c0d5ab0 unknown :-1 s=CLOSED pgs=298 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:19.480 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.476+0000 7f8d5e6556c0 1 --2- 10.20.193.27:0/3603816533 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8d58122ad0 0x7f8d58122f70 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:19.480 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.476+0000 7f8d5e6556c0 1 --2- 10.20.193.27:0/3603816533 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f8d58115410 0x7f8d5811d710 unknown :-1 s=CLOSED pgs=469 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:19.480 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.476+0000 7f8d5e6556c0 1 --2- 10.20.193.27:0/3603816533 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f8d5810f200 0x7f8d5811d1d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:19.480 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.477+0000 7f8d5e6556c0 1 -- 10.20.193.27:0/3603816533 >> 10.20.193.27:0/3603816533 conn(0x7f8d580ecb00 msgr2=0x7f8d5810cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:19.480 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.477+0000 7f8d5e6556c0 1 -- 10.20.193.27:0/3603816533 shutdown_connections 2026-02-13T02:26:19.480 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:19.477+0000 7f8d5e6556c0 1 -- 10.20.193.27:0/3603816533 wait complete. 2026-02-13T02:26:19.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:19 trial028 ceph-mon[26701]: pgmap v310: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:26:19.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:19 trial107 ceph-mon[27422]: pgmap v310: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:26:20.401 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:20 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3603816533' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:20.401 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:20 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:26:20.401 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:20 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:26:20.401 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:20 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:26:20.401 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:20 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:26:20.402 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:20 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:26:20.402 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:20 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:26:20.402 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:20 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:26:20.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:20 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3603816533' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:20.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:20 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:26:20.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:20 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:26:20.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:20 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:26:20.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:20 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:26:20.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:20 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:26:20.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:20 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:26:20.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:20 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:26:20.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:20 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3603816533' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:20.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:20 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:26:20.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:20 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:26:20.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:20 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:26:20.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:20 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:26:20.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:20 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:26:20.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:20 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:26:20.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:20 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:26:21.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:21 trial027 ceph-mon[24567]: pgmap v311: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:21.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:21 trial027 ceph-mon[24567]: pgmap v312: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 981 B/s rd, 1 op/s 2026-02-13T02:26:21.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:21 trial027 ceph-mon[24567]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:26:21.409 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:21 trial027 ceph-mon[24567]: Cluster is now healthy 2026-02-13T02:26:21.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:21 trial028 ceph-mon[26701]: pgmap v311: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:21.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:21 trial028 ceph-mon[26701]: pgmap v312: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 981 B/s rd, 1 op/s 2026-02-13T02:26:21.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:21 trial028 ceph-mon[26701]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:26:21.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:21 trial028 ceph-mon[26701]: Cluster is now healthy 2026-02-13T02:26:21.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:21 trial107 ceph-mon[27422]: pgmap v311: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:21.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:21 trial107 ceph-mon[27422]: pgmap v312: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 981 B/s rd, 1 op/s 2026-02-13T02:26:21.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:21 trial107 ceph-mon[27422]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:26:21.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:21 trial107 ceph-mon[27422]: Cluster is now healthy 2026-02-13T02:26:22.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:22 trial027 ceph-mon[24567]: pgmap v313: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:26:22.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:22 trial027 ceph-mon[24567]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:26:22.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:22 trial028 ceph-mon[26701]: pgmap v313: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:26:22.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:22 trial028 ceph-mon[26701]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:26:22.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:22 trial107 ceph-mon[27422]: pgmap v313: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:26:22.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:22 trial107 ceph-mon[27422]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:26:22.522 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:26:22.640 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:26:22.758 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.753+0000 7fb3a75356c0 1 -- 10.20.193.27:0/4221096694 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb3a010f200 msgr2=0x7fb3a010f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:22.758 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.753+0000 7fb3a75356c0 1 --2- 10.20.193.27:0/4221096694 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb3a010f200 0x7fb3a010f5d0 secure :-1 s=READY pgs=470 gs=1 cs=0 l=1 c_cookie=12720602563222612158 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb3900099a0 tx=0x7fb39002ed20 comp rx=0 tx=0).stop 2026-02-13T02:26:22.758 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.753+0000 7fb3a75356c0 1 -- 10.20.193.27:0/4221096694 shutdown_connections 2026-02-13T02:26:22.758 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.753+0000 7fb3a75356c0 1 --2- 10.20.193.27:0/4221096694 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb3a0115410 0x7fb3a0175db0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:22.759 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.753+0000 7fb3a75356c0 1 --2- 10.20.193.27:0/4221096694 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb3a010fbf0 0x7fb3a0114c30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:22.759 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.753+0000 7fb3a75356c0 1 --2- 10.20.193.27:0/4221096694 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb3a010f200 0x7fb3a010f5d0 unknown :-1 s=CLOSED pgs=470 gs=1 cs=0 l=1 c_cookie=12720602563222612158 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:22.759 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.753+0000 7fb3a75356c0 1 -- 10.20.193.27:0/4221096694 >> 10.20.193.27:0/4221096694 conn(0x7fb3a00ecb00 msgr2=0x7fb3a010cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:22.759 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.753+0000 7fb3a75356c0 1 -- 10.20.193.27:0/4221096694 shutdown_connections 2026-02-13T02:26:22.759 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.753+0000 7fb3a75356c0 1 -- 10.20.193.27:0/4221096694 wait complete. 2026-02-13T02:26:22.759 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.754+0000 7fb3a75356c0 1 Processor -- start 2026-02-13T02:26:22.759 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.754+0000 7fb3a75356c0 1 -- start start 2026-02-13T02:26:22.759 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.754+0000 7fb3a75356c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb3a010f200 0x7fb3a011d350 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:22.759 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.754+0000 7fb3a75356c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb3a0115410 0x7fb3a011d890 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:22.759 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.754+0000 7fb3a75356c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb3a0122c50 0x7fb3a01230f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:22.759 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.754+0000 7fb3a75356c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a0123660 con 0x7fb3a0122c50 2026-02-13T02:26:22.760 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.754+0000 7fb3a75356c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a01237d0 con 0x7fb3a010f200 2026-02-13T02:26:22.760 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.754+0000 7fb3a75356c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a0123990 con 0x7fb3a0115410 2026-02-13T02:26:22.760 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.754+0000 7fb3a5aa76c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb3a0122c50 0x7fb3a01230f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:22.760 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.754+0000 7fb3a5aa76c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb3a0122c50 0x7fb3a01230f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:35212/0 (socket says 10.20.193.27:35212) 2026-02-13T02:26:22.760 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.754+0000 7fb3a5aa76c0 1 -- 10.20.193.27:0/996085144 learned_addr learned my addr 10.20.193.27:0/996085144 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:26:22.760 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.754+0000 7fb3a4aa56c0 1 --2- 10.20.193.27:0/996085144 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb3a0115410 0x7fb3a011d890 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:22.760 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.754+0000 7fb3a5aa76c0 1 -- 10.20.193.27:0/996085144 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb3a0115410 msgr2=0x7fb3a011d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:22.760 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.754+0000 7fb3a5aa76c0 1 --2- 10.20.193.27:0/996085144 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb3a0115410 0x7fb3a011d890 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:22.760 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.754+0000 7fb3a5aa76c0 1 -- 10.20.193.27:0/996085144 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb3a010f200 msgr2=0x7fb3a011d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:22.760 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.754+0000 7fb3a52a66c0 1 --2- 10.20.193.27:0/996085144 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb3a010f200 0x7fb3a011d350 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:22.760 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.754+0000 7fb3a5aa76c0 1 --2- 10.20.193.27:0/996085144 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb3a010f200 0x7fb3a011d350 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:22.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.754+0000 7fb3a5aa76c0 1 -- 10.20.193.27:0/996085144 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb390009590 con 0x7fb3a0122c50 2026-02-13T02:26:22.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.754+0000 7fb3a52a66c0 1 --2- 10.20.193.27:0/996085144 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb3a010f200 0x7fb3a011d350 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:26:22.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.754+0000 7fb3a5aa76c0 1 --2- 10.20.193.27:0/996085144 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb3a0122c50 0x7fb3a01230f0 secure :-1 s=READY pgs=471 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb39c006ea0 tx=0x7fb39c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:22.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.754+0000 7fb38e7fc6c0 1 -- 10.20.193.27:0/996085144 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb39c003a30 con 0x7fb3a0122c50 2026-02-13T02:26:22.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.754+0000 7fb38e7fc6c0 1 -- 10.20.193.27:0/996085144 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb39c0054f0 con 0x7fb3a0122c50 2026-02-13T02:26:22.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.754+0000 7fb3a75356c0 1 -- 10.20.193.27:0/996085144 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3a0163e40 con 0x7fb3a0122c50 2026-02-13T02:26:22.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.754+0000 7fb38e7fc6c0 1 -- 10.20.193.27:0/996085144 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb39c00dcd0 con 0x7fb3a0122c50 2026-02-13T02:26:22.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.754+0000 7fb3a75356c0 1 -- 10.20.193.27:0/996085144 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3a01642d0 con 0x7fb3a0122c50 2026-02-13T02:26:22.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.755+0000 7fb3a75356c0 1 -- 10.20.193.27:0/996085144 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3a01104d0 con 0x7fb3a0122c50 2026-02-13T02:26:22.762 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.756+0000 7fb38e7fc6c0 1 -- 10.20.193.27:0/996085144 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7fb39c005030 con 0x7fb3a0122c50 2026-02-13T02:26:22.762 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.757+0000 7fb38e7fc6c0 1 --2- 10.20.193.27:0/996085144 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb37c0b5600 0x7fb37c0d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:22.762 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.757+0000 7fb3a52a66c0 1 --2- 10.20.193.27:0/996085144 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb37c0b5600 0x7fb37c0d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:22.763 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.757+0000 7fb38e7fc6c0 1 -- 10.20.193.27:0/996085144 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7fb39c0e4b70 con 0x7fb3a0122c50 2026-02-13T02:26:22.763 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.758+0000 7fb38e7fc6c0 1 -- 10.20.193.27:0/996085144 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fb39c0a75f0 con 0x7fb3a0122c50 2026-02-13T02:26:22.763 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.758+0000 7fb3a52a66c0 1 --2- 10.20.193.27:0/996085144 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb37c0b5600 0x7fb37c0d5ab0 secure :-1 s=READY pgs=299 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb3900098c0 tx=0x7fb39002f580 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:22.878 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.876+0000 7fb3a75356c0 1 -- 10.20.193.27:0/996085144 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb3a010f5d0 con 0x7fb3a0122c50 2026-02-13T02:26:22.879 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.876+0000 7fb38e7fc6c0 1 -- 10.20.193.27:0/996085144 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7fb39c0aa7a0 con 0x7fb3a0122c50 2026-02-13T02:26:22.879 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:26:22.880 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:26:22.881 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.878+0000 7fb3a75356c0 1 -- 10.20.193.27:0/996085144 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb37c0b5600 msgr2=0x7fb37c0d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:22.881 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.878+0000 7fb3a75356c0 1 --2- 10.20.193.27:0/996085144 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb37c0b5600 0x7fb37c0d5ab0 secure :-1 s=READY pgs=299 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb3900098c0 tx=0x7fb39002f580 comp rx=0 tx=0).stop 2026-02-13T02:26:22.881 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.878+0000 7fb3a75356c0 1 -- 10.20.193.27:0/996085144 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb3a0122c50 msgr2=0x7fb3a01230f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:22.881 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.878+0000 7fb3a75356c0 1 --2- 10.20.193.27:0/996085144 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb3a0122c50 0x7fb3a01230f0 secure :-1 s=READY pgs=471 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb39c006ea0 tx=0x7fb39c00cca0 comp rx=0 tx=0).stop 2026-02-13T02:26:22.881 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.878+0000 7fb3a75356c0 1 -- 10.20.193.27:0/996085144 shutdown_connections 2026-02-13T02:26:22.881 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.878+0000 7fb3a75356c0 1 --2- 10.20.193.27:0/996085144 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb37c0b5600 0x7fb37c0d5ab0 unknown :-1 s=CLOSED pgs=299 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:22.881 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.878+0000 7fb3a75356c0 1 --2- 10.20.193.27:0/996085144 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb3a0122c50 0x7fb3a01230f0 unknown :-1 s=CLOSED pgs=471 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:22.881 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.878+0000 7fb3a75356c0 1 --2- 10.20.193.27:0/996085144 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb3a0115410 0x7fb3a011d890 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:22.882 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.878+0000 7fb3a75356c0 1 --2- 10.20.193.27:0/996085144 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb3a010f200 0x7fb3a011d350 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:22.882 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.878+0000 7fb3a75356c0 1 -- 10.20.193.27:0/996085144 >> 10.20.193.27:0/996085144 conn(0x7fb3a00ecb00 msgr2=0x7fb3a010cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:22.882 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.878+0000 7fb3a75356c0 1 -- 10.20.193.27:0/996085144 shutdown_connections 2026-02-13T02:26:22.882 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:22.878+0000 7fb3a75356c0 1 -- 10.20.193.27:0/996085144 wait complete. 2026-02-13T02:26:23.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:23 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/996085144' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:23.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:23 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/996085144' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:23.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:23 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/996085144' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:24 trial027 ceph-mon[24567]: pgmap v314: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:26:24.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:24 trial028 ceph-mon[26701]: pgmap v314: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:26:24.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:24 trial107 ceph-mon[27422]: pgmap v314: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:26:25.922 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:26:26.040 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:26:26.161 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.156+0000 7fd8254106c0 1 -- 10.20.193.27:0/1222946769 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd82010f200 msgr2=0x7fd82010f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:26.161 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.156+0000 7fd8254106c0 1 --2- 10.20.193.27:0/1222946769 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd82010f200 0x7fd82010f5d0 secure :-1 s=READY pgs=472 gs=1 cs=0 l=1 c_cookie=7906430878864838108 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd8140099a0 tx=0x7fd81402ed20 comp rx=0 tx=0).stop 2026-02-13T02:26:26.161 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.157+0000 7fd8254106c0 1 -- 10.20.193.27:0/1222946769 shutdown_connections 2026-02-13T02:26:26.161 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.157+0000 7fd8254106c0 1 --2- 10.20.193.27:0/1222946769 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd820115410 0x7fd820175db0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:26.162 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.157+0000 7fd8254106c0 1 --2- 10.20.193.27:0/1222946769 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd82010fbf0 0x7fd820114c30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:26.162 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.157+0000 7fd8254106c0 1 --2- 10.20.193.27:0/1222946769 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd82010f200 0x7fd82010f5d0 unknown :-1 s=CLOSED pgs=472 gs=1 cs=0 l=1 c_cookie=7906430878864838108 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:26.162 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.157+0000 7fd8254106c0 1 -- 10.20.193.27:0/1222946769 >> 10.20.193.27:0/1222946769 conn(0x7fd8200ecb00 msgr2=0x7fd82010cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:26.162 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.157+0000 7fd8254106c0 1 -- 10.20.193.27:0/1222946769 shutdown_connections 2026-02-13T02:26:26.162 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.157+0000 7fd8254106c0 1 -- 10.20.193.27:0/1222946769 wait complete. 2026-02-13T02:26:26.162 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.157+0000 7fd8254106c0 1 Processor -- start 2026-02-13T02:26:26.162 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.157+0000 7fd8254106c0 1 -- start start 2026-02-13T02:26:26.162 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.157+0000 7fd8254106c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd82010f200 0x7fd82011d260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:26.163 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.157+0000 7fd8254106c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd820115410 0x7fd82011d7a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:26.163 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.157+0000 7fd8254106c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd820122b60 0x7fd820123000 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:26.163 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.157+0000 7fd8254106c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd820123570 con 0x7fd820122b60 2026-02-13T02:26:26.163 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.157+0000 7fd8254106c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd8201236e0 con 0x7fd82010f200 2026-02-13T02:26:26.163 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.157+0000 7fd8254106c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd8201238a0 con 0x7fd820115410 2026-02-13T02:26:26.163 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.157+0000 7fd81f7fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd820122b60 0x7fd820123000 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:26.163 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.157+0000 7fd81f7fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd820122b60 0x7fd820123000 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:35234/0 (socket says 10.20.193.27:35234) 2026-02-13T02:26:26.163 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.157+0000 7fd81f7fe6c0 1 -- 10.20.193.27:0/3997683952 learned_addr learned my addr 10.20.193.27:0/3997683952 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:26:26.163 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.157+0000 7fd81e7fc6c0 1 --2- 10.20.193.27:0/3997683952 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd820115410 0x7fd82011d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:26.163 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.157+0000 7fd81effd6c0 1 --2- 10.20.193.27:0/3997683952 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd82010f200 0x7fd82011d260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:26.163 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.157+0000 7fd81f7fe6c0 1 -- 10.20.193.27:0/3997683952 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd820115410 msgr2=0x7fd82011d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:26.164 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.157+0000 7fd81f7fe6c0 1 --2- 10.20.193.27:0/3997683952 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd820115410 0x7fd82011d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:26.164 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.157+0000 7fd81f7fe6c0 1 -- 10.20.193.27:0/3997683952 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd82010f200 msgr2=0x7fd82011d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:26.164 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.157+0000 7fd81f7fe6c0 1 --2- 10.20.193.27:0/3997683952 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd82010f200 0x7fd82011d260 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:26.164 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.157+0000 7fd81f7fe6c0 1 -- 10.20.193.27:0/3997683952 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd814009590 con 0x7fd820122b60 2026-02-13T02:26:26.164 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.157+0000 7fd81e7fc6c0 1 --2- 10.20.193.27:0/3997683952 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd820115410 0x7fd82011d7a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:26:26.164 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.157+0000 7fd81f7fe6c0 1 --2- 10.20.193.27:0/3997683952 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd820122b60 0x7fd820123000 secure :-1 s=READY pgs=473 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd810006ea0 tx=0x7fd81000ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:26.164 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.158+0000 7fd7fffff6c0 1 -- 10.20.193.27:0/3997683952 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd810003c40 con 0x7fd820122b60 2026-02-13T02:26:26.164 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.158+0000 7fd7fffff6c0 1 -- 10.20.193.27:0/3997683952 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd810003da0 con 0x7fd820122b60 2026-02-13T02:26:26.164 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.158+0000 7fd7fffff6c0 1 -- 10.20.193.27:0/3997683952 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd8100081c0 con 0x7fd820122b60 2026-02-13T02:26:26.164 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.158+0000 7fd8254106c0 1 -- 10.20.193.27:0/3997683952 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd820163d50 con 0x7fd820122b60 2026-02-13T02:26:26.165 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.158+0000 7fd8254106c0 1 -- 10.20.193.27:0/3997683952 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd820164230 con 0x7fd820122b60 2026-02-13T02:26:26.165 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.158+0000 7fd8254106c0 1 -- 10.20.193.27:0/3997683952 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8201104d0 con 0x7fd820122b60 2026-02-13T02:26:26.166 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.160+0000 7fd7fffff6c0 1 -- 10.20.193.27:0/3997683952 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7fd810005860 con 0x7fd820122b60 2026-02-13T02:26:26.166 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.161+0000 7fd7fffff6c0 1 --2- 10.20.193.27:0/3997683952 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd7f40b5680 0x7fd7f40d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:26.166 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.161+0000 7fd81effd6c0 1 --2- 10.20.193.27:0/3997683952 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd7f40b5680 0x7fd7f40d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:26.166 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.161+0000 7fd7fffff6c0 1 -- 10.20.193.27:0/3997683952 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7fd8100e4b10 con 0x7fd820122b60 2026-02-13T02:26:26.166 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.161+0000 7fd81effd6c0 1 --2- 10.20.193.27:0/3997683952 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd7f40b5680 0x7fd7f40d5b30 secure :-1 s=READY pgs=300 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd8140023e0 tx=0x7fd814005c60 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:26.166 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.161+0000 7fd7fffff6c0 1 -- 10.20.193.27:0/3997683952 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fd8100a75d0 con 0x7fd820122b60 2026-02-13T02:26:26.282 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.280+0000 7fd8254106c0 1 -- 10.20.193.27:0/3997683952 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd82010f5d0 con 0x7fd820122b60 2026-02-13T02:26:26.283 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:26 trial027 ceph-mon[24567]: pgmap v315: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:26:26.283 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.280+0000 7fd7fffff6c0 1 -- 10.20.193.27:0/3997683952 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7fd8100aa780 con 0x7fd820122b60 2026-02-13T02:26:26.283 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:26:26.284 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:26:26.285 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.282+0000 7fd8254106c0 1 -- 10.20.193.27:0/3997683952 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd7f40b5680 msgr2=0x7fd7f40d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:26.285 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.282+0000 7fd8254106c0 1 --2- 10.20.193.27:0/3997683952 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd7f40b5680 0x7fd7f40d5b30 secure :-1 s=READY pgs=300 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd8140023e0 tx=0x7fd814005c60 comp rx=0 tx=0).stop 2026-02-13T02:26:26.285 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.282+0000 7fd8254106c0 1 -- 10.20.193.27:0/3997683952 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd820122b60 msgr2=0x7fd820123000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:26.285 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.282+0000 7fd8254106c0 1 --2- 10.20.193.27:0/3997683952 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd820122b60 0x7fd820123000 secure :-1 s=READY pgs=473 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd810006ea0 tx=0x7fd81000ce50 comp rx=0 tx=0).stop 2026-02-13T02:26:26.285 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.282+0000 7fd8254106c0 1 -- 10.20.193.27:0/3997683952 shutdown_connections 2026-02-13T02:26:26.285 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.282+0000 7fd8254106c0 1 --2- 10.20.193.27:0/3997683952 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd7f40b5680 0x7fd7f40d5b30 unknown :-1 s=CLOSED pgs=300 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:26.285 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.282+0000 7fd8254106c0 1 --2- 10.20.193.27:0/3997683952 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd820122b60 0x7fd820123000 unknown :-1 s=CLOSED pgs=473 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:26.285 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.282+0000 7fd8254106c0 1 --2- 10.20.193.27:0/3997683952 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd820115410 0x7fd82011d7a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:26.286 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.282+0000 7fd8254106c0 1 --2- 10.20.193.27:0/3997683952 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd82010f200 0x7fd82011d260 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:26.286 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.282+0000 7fd8254106c0 1 -- 10.20.193.27:0/3997683952 >> 10.20.193.27:0/3997683952 conn(0x7fd8200ecb00 msgr2=0x7fd82010cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:26.286 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.282+0000 7fd8254106c0 1 -- 10.20.193.27:0/3997683952 shutdown_connections 2026-02-13T02:26:26.286 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:26.282+0000 7fd8254106c0 1 -- 10.20.193.27:0/3997683952 wait complete. 2026-02-13T02:26:26.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:26 trial028 ceph-mon[26701]: pgmap v315: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:26:26.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:26 trial107 ceph-mon[27422]: pgmap v315: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:26:27.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:27 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3997683952' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:27.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:27 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3997683952' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:27.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:27 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3997683952' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:28.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:28 trial027 ceph-mon[24567]: pgmap v316: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 564 B/s rd, 1 op/s 2026-02-13T02:26:28.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:28 trial028 ceph-mon[26701]: pgmap v316: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 564 B/s rd, 1 op/s 2026-02-13T02:26:28.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:28 trial107 ceph-mon[27422]: pgmap v316: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 564 B/s rd, 1 op/s 2026-02-13T02:26:29.327 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:26:29.447 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:26:29.564 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.559+0000 7f63fe9c76c0 1 -- 10.20.193.27:0/3548287804 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f63f8115410 msgr2=0x7f63f8175db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:29.564 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.559+0000 7f63fe9c76c0 1 --2- 10.20.193.27:0/3548287804 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f63f8115410 0x7f63f8175db0 secure :-1 s=READY pgs=474 gs=1 cs=0 l=1 c_cookie=7258252215118947239 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f63ec00aa70 tx=0x7f63ec02fde0 comp rx=0 tx=0).stop 2026-02-13T02:26:29.564 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.559+0000 7f63fe9c76c0 1 -- 10.20.193.27:0/3548287804 shutdown_connections 2026-02-13T02:26:29.565 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.559+0000 7f63fe9c76c0 1 --2- 10.20.193.27:0/3548287804 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f63f8115410 0x7f63f8175db0 unknown :-1 s=CLOSED pgs=474 gs=1 cs=0 l=1 c_cookie=7258252215118947239 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:29.565 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.559+0000 7f63fe9c76c0 1 --2- 10.20.193.27:0/3548287804 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f63f810fbf0 0x7f63f8114c30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:29.565 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.559+0000 7f63fe9c76c0 1 --2- 10.20.193.27:0/3548287804 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f63f810f200 0x7f63f810f5d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:29.565 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.559+0000 7f63fe9c76c0 1 -- 10.20.193.27:0/3548287804 >> 10.20.193.27:0/3548287804 conn(0x7f63f80ecb00 msgr2=0x7f63f810cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:29.565 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.560+0000 7f63fe9c76c0 1 -- 10.20.193.27:0/3548287804 shutdown_connections 2026-02-13T02:26:29.565 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.560+0000 7f63fe9c76c0 1 -- 10.20.193.27:0/3548287804 wait complete. 2026-02-13T02:26:29.565 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.560+0000 7f63fe9c76c0 1 Processor -- start 2026-02-13T02:26:29.565 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.560+0000 7f63fe9c76c0 1 -- start start 2026-02-13T02:26:29.565 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.560+0000 7f63fe9c76c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f63f810f200 0x7f63f811d2b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:29.565 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.560+0000 7f63fe9c76c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f63f8115410 0x7f63f811d7f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:29.566 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.560+0000 7f63fe9c76c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f63f8122bb0 0x7f63f8123050 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:29.566 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.560+0000 7f63fe9c76c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f63f8123590 con 0x7f63f8122bb0 2026-02-13T02:26:29.566 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.560+0000 7f63fe9c76c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f63f8123700 con 0x7f63f8115410 2026-02-13T02:26:29.566 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.560+0000 7f63fe9c76c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f63f81238c0 con 0x7f63f810f200 2026-02-13T02:26:29.566 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.560+0000 7f63fcf396c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f63f8122bb0 0x7f63f8123050 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:29.566 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.560+0000 7f63fcf396c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f63f8122bb0 0x7f63f8123050 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:56772/0 (socket says 10.20.193.27:56772) 2026-02-13T02:26:29.566 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.560+0000 7f63fcf396c0 1 -- 10.20.193.27:0/2298813323 learned_addr learned my addr 10.20.193.27:0/2298813323 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:26:29.566 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.560+0000 7f63f77fe6c0 1 --2- 10.20.193.27:0/2298813323 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f63f8115410 0x7f63f811d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:29.566 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.560+0000 7f63f7fff6c0 1 --2- 10.20.193.27:0/2298813323 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f63f810f200 0x7f63f811d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:29.566 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.560+0000 7f63fcf396c0 1 -- 10.20.193.27:0/2298813323 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f63f810f200 msgr2=0x7f63f811d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:29.566 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.560+0000 7f63fcf396c0 1 --2- 10.20.193.27:0/2298813323 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f63f810f200 0x7f63f811d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:29.566 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.560+0000 7f63fcf396c0 1 -- 10.20.193.27:0/2298813323 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f63f8115410 msgr2=0x7f63f811d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:29.567 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.560+0000 7f63fcf396c0 1 --2- 10.20.193.27:0/2298813323 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f63f8115410 0x7f63f811d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:29.567 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.560+0000 7f63fcf396c0 1 -- 10.20.193.27:0/2298813323 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ec00a660 con 0x7f63f8122bb0 2026-02-13T02:26:29.567 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.560+0000 7f63fcf396c0 1 --2- 10.20.193.27:0/2298813323 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f63f8122bb0 0x7f63f8123050 secure :-1 s=READY pgs=475 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f63ec0054b0 tx=0x7f63ec0054e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:29.567 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.560+0000 7f63f57fa6c0 1 -- 10.20.193.27:0/2298813323 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f63ec003c40 con 0x7f63f8122bb0 2026-02-13T02:26:29.567 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.561+0000 7f63fe9c76c0 1 -- 10.20.193.27:0/2298813323 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63f8163d40 con 0x7f63f8122bb0 2026-02-13T02:26:29.567 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.561+0000 7f63f57fa6c0 1 -- 10.20.193.27:0/2298813323 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f63ec003da0 con 0x7f63f8122bb0 2026-02-13T02:26:29.567 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.561+0000 7f63f57fa6c0 1 -- 10.20.193.27:0/2298813323 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f63ec0382f0 con 0x7f63f8122bb0 2026-02-13T02:26:29.567 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.561+0000 7f63fe9c76c0 1 -- 10.20.193.27:0/2298813323 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63f81641c0 con 0x7f63f8122bb0 2026-02-13T02:26:29.567 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.561+0000 7f63fe9c76c0 1 -- 10.20.193.27:0/2298813323 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63f81104d0 con 0x7f63f8122bb0 2026-02-13T02:26:29.568 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.562+0000 7f63f57fa6c0 1 -- 10.20.193.27:0/2298813323 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f63ec008260 con 0x7f63f8122bb0 2026-02-13T02:26:29.569 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.563+0000 7f63f57fa6c0 1 --2- 10.20.193.27:0/2298813323 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f63cc0b5600 0x7f63cc0d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:29.569 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.563+0000 7f63f7fff6c0 1 --2- 10.20.193.27:0/2298813323 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f63cc0b5600 0x7f63cc0d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:29.569 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.563+0000 7f63f57fa6c0 1 -- 10.20.193.27:0/2298813323 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f63ec108db0 con 0x7f63f8122bb0 2026-02-13T02:26:29.569 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.563+0000 7f63f7fff6c0 1 --2- 10.20.193.27:0/2298813323 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f63cc0b5600 0x7f63cc0d5ab0 secure :-1 s=READY pgs=301 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f63e4004750 tx=0x7f63e4009260 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:29.569 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.564+0000 7f63f57fa6c0 1 -- 10.20.193.27:0/2298813323 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f63ec0cb830 con 0x7f63f8122bb0 2026-02-13T02:26:29.684 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.682+0000 7f63fe9c76c0 1 -- 10.20.193.27:0/2298813323 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f63f810f5d0 con 0x7f63f8122bb0 2026-02-13T02:26:29.684 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.682+0000 7f63f57fa6c0 1 -- 10.20.193.27:0/2298813323 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f63ec0ce9e0 con 0x7f63f8122bb0 2026-02-13T02:26:29.684 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:26:29.685 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:26:29.686 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.683+0000 7f63fe9c76c0 1 -- 10.20.193.27:0/2298813323 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f63cc0b5600 msgr2=0x7f63cc0d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:29.686 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.683+0000 7f63fe9c76c0 1 --2- 10.20.193.27:0/2298813323 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f63cc0b5600 0x7f63cc0d5ab0 secure :-1 s=READY pgs=301 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f63e4004750 tx=0x7f63e4009260 comp rx=0 tx=0).stop 2026-02-13T02:26:29.686 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.683+0000 7f63fe9c76c0 1 -- 10.20.193.27:0/2298813323 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f63f8122bb0 msgr2=0x7f63f8123050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:29.686 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.683+0000 7f63fe9c76c0 1 --2- 10.20.193.27:0/2298813323 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f63f8122bb0 0x7f63f8123050 secure :-1 s=READY pgs=475 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f63ec0054b0 tx=0x7f63ec0054e0 comp rx=0 tx=0).stop 2026-02-13T02:26:29.686 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.683+0000 7f63fe9c76c0 1 -- 10.20.193.27:0/2298813323 shutdown_connections 2026-02-13T02:26:29.686 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.683+0000 7f63fe9c76c0 1 --2- 10.20.193.27:0/2298813323 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f63cc0b5600 0x7f63cc0d5ab0 unknown :-1 s=CLOSED pgs=301 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:29.686 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.683+0000 7f63fe9c76c0 1 --2- 10.20.193.27:0/2298813323 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f63f8122bb0 0x7f63f8123050 unknown :-1 s=CLOSED pgs=475 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:29.686 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.683+0000 7f63fe9c76c0 1 --2- 10.20.193.27:0/2298813323 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f63f8115410 0x7f63f811d7f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:29.687 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.683+0000 7f63fe9c76c0 1 --2- 10.20.193.27:0/2298813323 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f63f810f200 0x7f63f811d2b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:29.687 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.683+0000 7f63fe9c76c0 1 -- 10.20.193.27:0/2298813323 >> 10.20.193.27:0/2298813323 conn(0x7f63f80ecb00 msgr2=0x7f63f810cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:29.687 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.683+0000 7f63fe9c76c0 1 -- 10.20.193.27:0/2298813323 shutdown_connections 2026-02-13T02:26:29.687 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:29.683+0000 7f63fe9c76c0 1 -- 10.20.193.27:0/2298813323 wait complete. 2026-02-13T02:26:30.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:30 trial027 ceph-mon[24567]: pgmap v317: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:26:30.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:30 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2298813323' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:30.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:30 trial028 ceph-mon[26701]: pgmap v317: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:26:30.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:30 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2298813323' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:30.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:30 trial107 ceph-mon[27422]: pgmap v317: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:26:30.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:30 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2298813323' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:32.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:32 trial027 ceph-mon[24567]: pgmap v318: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 963 B/s rd, 1 op/s 2026-02-13T02:26:32.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:32 trial028 ceph-mon[26701]: pgmap v318: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 963 B/s rd, 1 op/s 2026-02-13T02:26:32.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:32 trial107 ceph-mon[27422]: pgmap v318: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 963 B/s rd, 1 op/s 2026-02-13T02:26:32.727 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:26:32.847 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:26:32.965 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.960+0000 7f023277d6c0 1 -- 10.20.193.27:0/2083346469 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f022c10f200 msgr2=0x7f022c10f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:32.965 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.960+0000 7f023277d6c0 1 --2- 10.20.193.27:0/2083346469 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f022c10f200 0x7f022c10f5d0 secure :-1 s=READY pgs=476 gs=1 cs=0 l=1 c_cookie=16478585646207700279 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f02180099a0 tx=0x7f021802ed20 comp rx=0 tx=0).stop 2026-02-13T02:26:32.965 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.960+0000 7f023277d6c0 1 -- 10.20.193.27:0/2083346469 shutdown_connections 2026-02-13T02:26:32.965 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.960+0000 7f023277d6c0 1 --2- 10.20.193.27:0/2083346469 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f022c115410 0x7f022c175db0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:32.965 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.960+0000 7f023277d6c0 1 --2- 10.20.193.27:0/2083346469 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f022c10fbf0 0x7f022c114c30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:32.965 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.960+0000 7f023277d6c0 1 --2- 10.20.193.27:0/2083346469 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f022c10f200 0x7f022c10f5d0 unknown :-1 s=CLOSED pgs=476 gs=1 cs=0 l=1 c_cookie=16478585646207700279 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:32.965 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.960+0000 7f023277d6c0 1 -- 10.20.193.27:0/2083346469 >> 10.20.193.27:0/2083346469 conn(0x7f022c0ecb00 msgr2=0x7f022c10cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:32.966 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.960+0000 7f023277d6c0 1 -- 10.20.193.27:0/2083346469 shutdown_connections 2026-02-13T02:26:32.966 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.960+0000 7f023277d6c0 1 -- 10.20.193.27:0/2083346469 wait complete. 2026-02-13T02:26:32.966 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.961+0000 7f023277d6c0 1 Processor -- start 2026-02-13T02:26:32.966 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.961+0000 7f023277d6c0 1 -- start start 2026-02-13T02:26:32.966 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.961+0000 7f023277d6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f022c10f200 0x7f022c11d350 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:32.966 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.961+0000 7f023277d6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f022c115410 0x7f022c11d890 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:32.966 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.961+0000 7f023277d6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f022c122c50 0x7f022c1230f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:32.966 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.961+0000 7f023277d6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f022c123660 con 0x7f022c10f200 2026-02-13T02:26:32.966 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.961+0000 7f023277d6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f022c1237d0 con 0x7f022c115410 2026-02-13T02:26:32.966 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.961+0000 7f023277d6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f022c123990 con 0x7f022c122c50 2026-02-13T02:26:32.966 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.961+0000 7f022bfff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f022c10f200 0x7f022c11d350 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:32.966 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.961+0000 7f022bfff6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f022c10f200 0x7f022c11d350 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:56792/0 (socket says 10.20.193.27:56792) 2026-02-13T02:26:32.967 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.961+0000 7f022bfff6c0 1 -- 10.20.193.27:0/2950976021 learned_addr learned my addr 10.20.193.27:0/2950976021 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:26:32.967 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.961+0000 7f022b7fe6c0 1 --2- 10.20.193.27:0/2950976021 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f022c115410 0x7f022c11d890 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:32.967 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.961+0000 7f0230cef6c0 1 --2- 10.20.193.27:0/2950976021 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f022c122c50 0x7f022c1230f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:32.967 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.961+0000 7f022bfff6c0 1 -- 10.20.193.27:0/2950976021 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f022c122c50 msgr2=0x7f022c1230f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:32.967 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.961+0000 7f022bfff6c0 1 --2- 10.20.193.27:0/2950976021 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f022c122c50 0x7f022c1230f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:32.967 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.961+0000 7f022bfff6c0 1 -- 10.20.193.27:0/2950976021 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f022c115410 msgr2=0x7f022c11d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:32.967 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.961+0000 7f022bfff6c0 1 --2- 10.20.193.27:0/2950976021 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f022c115410 0x7f022c11d890 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:32.967 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.961+0000 7f022bfff6c0 1 -- 10.20.193.27:0/2950976021 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0218009590 con 0x7f022c10f200 2026-02-13T02:26:32.967 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.961+0000 7f022b7fe6c0 1 --2- 10.20.193.27:0/2950976021 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f022c115410 0x7f022c11d890 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:26:32.967 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.961+0000 7f0230cef6c0 1 --2- 10.20.193.27:0/2950976021 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f022c122c50 0x7f022c1230f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:26:32.967 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.961+0000 7f022bfff6c0 1 --2- 10.20.193.27:0/2950976021 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f022c10f200 0x7f022c11d350 secure :-1 s=READY pgs=477 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f021802f230 tx=0x7f021802fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:32.967 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.961+0000 7f02297fa6c0 1 -- 10.20.193.27:0/2950976021 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0218002ba0 con 0x7f022c10f200 2026-02-13T02:26:32.968 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.961+0000 7f02297fa6c0 1 -- 10.20.193.27:0/2950976021 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f0218002d00 con 0x7f022c10f200 2026-02-13T02:26:32.968 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.961+0000 7f023277d6c0 1 -- 10.20.193.27:0/2950976021 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f022c163de0 con 0x7f022c10f200 2026-02-13T02:26:32.968 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.961+0000 7f02297fa6c0 1 -- 10.20.193.27:0/2950976021 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f02180372f0 con 0x7f022c10f200 2026-02-13T02:26:32.968 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.961+0000 7f023277d6c0 1 -- 10.20.193.27:0/2950976021 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f022c164260 con 0x7f022c10f200 2026-02-13T02:26:32.968 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.962+0000 7f023277d6c0 1 -- 10.20.193.27:0/2950976021 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f022c1104d0 con 0x7f022c10f200 2026-02-13T02:26:32.969 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.963+0000 7f02297fa6c0 1 -- 10.20.193.27:0/2950976021 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f02180078d0 con 0x7f022c10f200 2026-02-13T02:26:32.969 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.964+0000 7f02297fa6c0 1 --2- 10.20.193.27:0/2950976021 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f02000b5680 0x7f02000d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:32.969 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.964+0000 7f022b7fe6c0 1 --2- 10.20.193.27:0/2950976021 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f02000b5680 0x7f02000d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:32.969 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.964+0000 7f02297fa6c0 1 -- 10.20.193.27:0/2950976021 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f0218107ba0 con 0x7f022c10f200 2026-02-13T02:26:32.969 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.964+0000 7f022b7fe6c0 1 --2- 10.20.193.27:0/2950976021 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f02000b5680 0x7f02000d5b30 secure :-1 s=READY pgs=302 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f022c123ce0 tx=0x7f021c005f10 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:32.969 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:32.964+0000 7f02297fa6c0 1 -- 10.20.193.27:0/2950976021 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f02180ca620 con 0x7f022c10f200 2026-02-13T02:26:33.084 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:33.082+0000 7f023277d6c0 1 -- 10.20.193.27:0/2950976021 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f022c10f5d0 con 0x7f022c10f200 2026-02-13T02:26:33.084 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:33.082+0000 7f02297fa6c0 1 -- 10.20.193.27:0/2950976021 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f02180cd7d0 con 0x7f022c10f200 2026-02-13T02:26:33.085 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:26:33.085 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:26:33.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:33.084+0000 7f023277d6c0 1 -- 10.20.193.27:0/2950976021 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f02000b5680 msgr2=0x7f02000d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:33.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:33.084+0000 7f023277d6c0 1 --2- 10.20.193.27:0/2950976021 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f02000b5680 0x7f02000d5b30 secure :-1 s=READY pgs=302 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f022c123ce0 tx=0x7f021c005f10 comp rx=0 tx=0).stop 2026-02-13T02:26:33.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:33.084+0000 7f023277d6c0 1 -- 10.20.193.27:0/2950976021 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f022c10f200 msgr2=0x7f022c11d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:33.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:33.084+0000 7f023277d6c0 1 --2- 10.20.193.27:0/2950976021 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f022c10f200 0x7f022c11d350 secure :-1 s=READY pgs=477 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f021802f230 tx=0x7f021802fd40 comp rx=0 tx=0).stop 2026-02-13T02:26:33.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:33.084+0000 7f023277d6c0 1 -- 10.20.193.27:0/2950976021 shutdown_connections 2026-02-13T02:26:33.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:33.084+0000 7f023277d6c0 1 --2- 10.20.193.27:0/2950976021 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f02000b5680 0x7f02000d5b30 unknown :-1 s=CLOSED pgs=302 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:33.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:33.084+0000 7f023277d6c0 1 --2- 10.20.193.27:0/2950976021 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f022c122c50 0x7f022c1230f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:33.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:33.084+0000 7f023277d6c0 1 --2- 10.20.193.27:0/2950976021 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f022c115410 0x7f022c11d890 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:33.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:33.084+0000 7f023277d6c0 1 --2- 10.20.193.27:0/2950976021 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f022c10f200 0x7f022c11d350 unknown :-1 s=CLOSED pgs=477 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:33.087 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:33.084+0000 7f023277d6c0 1 -- 10.20.193.27:0/2950976021 >> 10.20.193.27:0/2950976021 conn(0x7f022c0ecb00 msgr2=0x7f022c10cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:33.088 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:33.084+0000 7f023277d6c0 1 -- 10.20.193.27:0/2950976021 shutdown_connections 2026-02-13T02:26:33.088 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:33.084+0000 7f023277d6c0 1 -- 10.20.193.27:0/2950976021 wait complete. 2026-02-13T02:26:33.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:33 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2950976021' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:33.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:33 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2950976021' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:33.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:33 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2950976021' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:34.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:34 trial027 ceph-mon[24567]: pgmap v319: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:26:34.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:34 trial028 ceph-mon[26701]: pgmap v319: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:26:34.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:34 trial107 ceph-mon[27422]: pgmap v319: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:26:36.127 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:26:36.247 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:26:36.368 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.364+0000 7f32f32d86c0 1 -- 10.20.193.27:0/3615956966 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f32ec10fc20 msgr2=0x7f32ec114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:36.369 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.364+0000 7f32f32d86c0 1 --2- 10.20.193.27:0/3615956966 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f32ec10fc20 0x7f32ec114c60 secure :-1 s=READY pgs=478 gs=1 cs=0 l=1 c_cookie=7645491143960467956 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f32e00099a0 tx=0x7f32e002ed20 comp rx=0 tx=0).stop 2026-02-13T02:26:36.369 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.364+0000 7f32f32d86c0 1 -- 10.20.193.27:0/3615956966 shutdown_connections 2026-02-13T02:26:36.369 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.364+0000 7f32f32d86c0 1 --2- 10.20.193.27:0/3615956966 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f32ec115440 0x7f32ec175de0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:36.369 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.364+0000 7f32f32d86c0 1 --2- 10.20.193.27:0/3615956966 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f32ec10fc20 0x7f32ec114c60 unknown :-1 s=CLOSED pgs=478 gs=1 cs=0 l=1 c_cookie=7645491143960467956 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:36.369 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.364+0000 7f32f32d86c0 1 --2- 10.20.193.27:0/3615956966 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f32ec10f230 0x7f32ec10f600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:36.369 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.364+0000 7f32f32d86c0 1 -- 10.20.193.27:0/3615956966 >> 10.20.193.27:0/3615956966 conn(0x7f32ec0ecb30 msgr2=0x7f32ec10cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:36.369 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.364+0000 7f32f32d86c0 1 -- 10.20.193.27:0/3615956966 shutdown_connections 2026-02-13T02:26:36.369 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.364+0000 7f32f32d86c0 1 -- 10.20.193.27:0/3615956966 wait complete. 2026-02-13T02:26:36.369 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.364+0000 7f32f32d86c0 1 Processor -- start 2026-02-13T02:26:36.369 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.364+0000 7f32f32d86c0 1 -- start start 2026-02-13T02:26:36.369 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.364+0000 7f32f32d86c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f32ec10f230 0x7f32ec0f4d00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:36.370 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.364+0000 7f32f32d86c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f32ec115440 0x7f32ec175ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:36.370 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.364+0000 7f32f32d86c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f32ec0f5330 0x7f32ec0fa5b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:36.370 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.364+0000 7f32f32d86c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec0f5a20 con 0x7f32ec0f5330 2026-02-13T02:26:36.370 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.364+0000 7f32f32d86c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec0faaf0 con 0x7f32ec115440 2026-02-13T02:26:36.370 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.364+0000 7f32f32d86c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec0facb0 con 0x7f32ec10f230 2026-02-13T02:26:36.370 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.364+0000 7f32f184a6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f32ec0f5330 0x7f32ec0fa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:36.370 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.364+0000 7f32f184a6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f32ec0f5330 0x7f32ec0fa5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:56812/0 (socket says 10.20.193.27:56812) 2026-02-13T02:26:36.370 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.364+0000 7f32f184a6c0 1 -- 10.20.193.27:0/528367223 learned_addr learned my addr 10.20.193.27:0/528367223 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:26:36.370 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.365+0000 7f32f10496c0 1 --2- 10.20.193.27:0/528367223 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f32ec10f230 0x7f32ec0f4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:36.370 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.365+0000 7f32f08486c0 1 --2- 10.20.193.27:0/528367223 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f32ec115440 0x7f32ec175ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:36.370 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.365+0000 7f32f184a6c0 1 -- 10.20.193.27:0/528367223 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f32ec10f230 msgr2=0x7f32ec0f4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:36.371 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.365+0000 7f32f184a6c0 1 --2- 10.20.193.27:0/528367223 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f32ec10f230 0x7f32ec0f4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:36.371 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.365+0000 7f32f184a6c0 1 -- 10.20.193.27:0/528367223 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f32ec115440 msgr2=0x7f32ec175ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:36.371 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.365+0000 7f32f184a6c0 1 --2- 10.20.193.27:0/528367223 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f32ec115440 0x7f32ec175ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:36.371 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.365+0000 7f32f184a6c0 1 -- 10.20.193.27:0/528367223 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e0009590 con 0x7f32ec0f5330 2026-02-13T02:26:36.371 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.365+0000 7f32f08486c0 1 --2- 10.20.193.27:0/528367223 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f32ec115440 0x7f32ec175ce0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:26:36.371 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.365+0000 7f32f184a6c0 1 --2- 10.20.193.27:0/528367223 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f32ec0f5330 0x7f32ec0fa5b0 secure :-1 s=READY pgs=479 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f32e8006ea0 tx=0x7f32e800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:36.371 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.365+0000 7f32da7fc6c0 1 -- 10.20.193.27:0/528367223 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f32e8003a30 con 0x7f32ec0f5330 2026-02-13T02:26:36.371 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.365+0000 7f32da7fc6c0 1 -- 10.20.193.27:0/528367223 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f32e80054f0 con 0x7f32ec0f5330 2026-02-13T02:26:36.371 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.365+0000 7f32f32d86c0 1 -- 10.20.193.27:0/528367223 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32ec0faf90 con 0x7f32ec0f5330 2026-02-13T02:26:36.371 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.365+0000 7f32da7fc6c0 1 -- 10.20.193.27:0/528367223 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f32e800dcd0 con 0x7f32ec0f5330 2026-02-13T02:26:36.371 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.365+0000 7f32f32d86c0 1 -- 10.20.193.27:0/528367223 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32ec0fb720 con 0x7f32ec0f5330 2026-02-13T02:26:36.371 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.365+0000 7f32f32d86c0 1 -- 10.20.193.27:0/528367223 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32ec0fb470 con 0x7f32ec0f5330 2026-02-13T02:26:36.372 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.368+0000 7f32da7fc6c0 1 -- 10.20.193.27:0/528367223 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f32e8003b90 con 0x7f32ec0f5330 2026-02-13T02:26:36.373 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.368+0000 7f32da7fc6c0 1 --2- 10.20.193.27:0/528367223 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f32c80b5600 0x7f32c80d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:36.373 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.368+0000 7f32da7fc6c0 1 -- 10.20.193.27:0/528367223 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f32e80e4ac0 con 0x7f32ec0f5330 2026-02-13T02:26:36.373 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.368+0000 7f32f10496c0 1 --2- 10.20.193.27:0/528367223 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f32c80b5600 0x7f32c80d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:36.373 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.368+0000 7f32da7fc6c0 1 -- 10.20.193.27:0/528367223 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f32e80a7580 con 0x7f32ec0f5330 2026-02-13T02:26:36.373 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.368+0000 7f32f10496c0 1 --2- 10.20.193.27:0/528367223 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f32c80b5600 0x7f32c80d5ab0 secure :-1 s=READY pgs=303 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f32dc005980 tx=0x7f32dc005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:36.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:36 trial027 ceph-mon[24567]: pgmap v320: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:36.484 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.483+0000 7f32f32d86c0 1 -- 10.20.193.27:0/528367223 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f32ec0f5e00 con 0x7f32ec0f5330 2026-02-13T02:26:36.484 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.483+0000 7f32da7fc6c0 1 -- 10.20.193.27:0/528367223 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f32e800de60 con 0x7f32ec0f5330 2026-02-13T02:26:36.485 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:26:36.485 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:26:36.486 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.483+0000 7f32f32d86c0 1 -- 10.20.193.27:0/528367223 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f32c80b5600 msgr2=0x7f32c80d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:36.486 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.483+0000 7f32f32d86c0 1 --2- 10.20.193.27:0/528367223 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f32c80b5600 0x7f32c80d5ab0 secure :-1 s=READY pgs=303 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f32dc005980 tx=0x7f32dc005910 comp rx=0 tx=0).stop 2026-02-13T02:26:36.486 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.483+0000 7f32f32d86c0 1 -- 10.20.193.27:0/528367223 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f32ec0f5330 msgr2=0x7f32ec0fa5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:36.486 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.483+0000 7f32f32d86c0 1 --2- 10.20.193.27:0/528367223 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f32ec0f5330 0x7f32ec0fa5b0 secure :-1 s=READY pgs=479 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f32e8006ea0 tx=0x7f32e800cca0 comp rx=0 tx=0).stop 2026-02-13T02:26:36.486 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.483+0000 7f32f32d86c0 1 -- 10.20.193.27:0/528367223 shutdown_connections 2026-02-13T02:26:36.486 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.483+0000 7f32f32d86c0 1 --2- 10.20.193.27:0/528367223 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f32c80b5600 0x7f32c80d5ab0 unknown :-1 s=CLOSED pgs=303 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:36.486 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.483+0000 7f32f32d86c0 1 --2- 10.20.193.27:0/528367223 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f32ec0f5330 0x7f32ec0fa5b0 unknown :-1 s=CLOSED pgs=479 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:36.487 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.483+0000 7f32f32d86c0 1 --2- 10.20.193.27:0/528367223 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f32ec115440 0x7f32ec175ce0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:36.487 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.483+0000 7f32f32d86c0 1 --2- 10.20.193.27:0/528367223 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f32ec10f230 0x7f32ec0f4d00 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:36.487 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.483+0000 7f32f32d86c0 1 -- 10.20.193.27:0/528367223 >> 10.20.193.27:0/528367223 conn(0x7f32ec0ecb30 msgr2=0x7f32ec10d970 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:36.487 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.483+0000 7f32f32d86c0 1 -- 10.20.193.27:0/528367223 shutdown_connections 2026-02-13T02:26:36.487 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:36.483+0000 7f32f32d86c0 1 -- 10.20.193.27:0/528367223 wait complete. 2026-02-13T02:26:36.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:36 trial028 ceph-mon[26701]: pgmap v320: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:36.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:36 trial107 ceph-mon[27422]: pgmap v320: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:37.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:37 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/528367223' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:37.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:37 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/528367223' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:37.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:37 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/528367223' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:38.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:38 trial028 ceph-mon[26701]: pgmap v321: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:38.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:38 trial107 ceph-mon[27422]: pgmap v321: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:38.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:38 trial027 ceph-mon[24567]: pgmap v321: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:39.526 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:26:39.646 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:26:39.760 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.757+0000 7fc8d76306c0 1 -- 10.20.193.27:0/72451918 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc8d010f230 msgr2=0x7fc8d010f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:39.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.757+0000 7fc8d76306c0 1 --2- 10.20.193.27:0/72451918 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc8d010f230 0x7fc8d010f600 secure :-1 s=READY pgs=480 gs=1 cs=0 l=1 c_cookie=12906917460545830500 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc8c00099a0 tx=0x7fc8c002ed20 comp rx=0 tx=0).stop 2026-02-13T02:26:39.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.757+0000 7fc8d76306c0 1 -- 10.20.193.27:0/72451918 shutdown_connections 2026-02-13T02:26:39.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.757+0000 7fc8d76306c0 1 --2- 10.20.193.27:0/72451918 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc8d0115440 0x7fc8d0175de0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:39.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.757+0000 7fc8d76306c0 1 --2- 10.20.193.27:0/72451918 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc8d010fc20 0x7fc8d0114c60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:39.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.757+0000 7fc8d76306c0 1 --2- 10.20.193.27:0/72451918 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc8d010f230 0x7fc8d010f600 unknown :-1 s=CLOSED pgs=480 gs=1 cs=0 l=1 c_cookie=12906917460545830500 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:39.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.757+0000 7fc8d76306c0 1 -- 10.20.193.27:0/72451918 >> 10.20.193.27:0/72451918 conn(0x7fc8d00ecb30 msgr2=0x7fc8d010cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:39.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.757+0000 7fc8d76306c0 1 -- 10.20.193.27:0/72451918 shutdown_connections 2026-02-13T02:26:39.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.757+0000 7fc8d76306c0 1 -- 10.20.193.27:0/72451918 wait complete. 2026-02-13T02:26:39.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.758+0000 7fc8d76306c0 1 Processor -- start 2026-02-13T02:26:39.761 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.758+0000 7fc8d76306c0 1 -- start start 2026-02-13T02:26:39.762 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.758+0000 7fc8d76306c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc8d010f230 0x7fc8d0066730 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:39.762 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.758+0000 7fc8d76306c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc8d0115440 0x7fc8d0062990 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:39.762 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.758+0000 7fc8d76306c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc8d0062f70 0x7fc8d0063410 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:39.762 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.758+0000 7fc8d76306c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d0167ca0 con 0x7fc8d010f230 2026-02-13T02:26:39.762 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.758+0000 7fc8d76306c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d0167e10 con 0x7fc8d0115440 2026-02-13T02:26:39.762 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.758+0000 7fc8d76306c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d0167fd0 con 0x7fc8d0062f70 2026-02-13T02:26:39.762 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.758+0000 7fc8d53a56c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc8d010f230 0x7fc8d0066730 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:39.762 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.758+0000 7fc8d53a56c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc8d010f230 0x7fc8d0066730 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55596/0 (socket says 10.20.193.27:55596) 2026-02-13T02:26:39.762 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.758+0000 7fc8d53a56c0 1 -- 10.20.193.27:0/714741578 learned_addr learned my addr 10.20.193.27:0/714741578 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:26:39.762 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.758+0000 7fc8d53a56c0 1 -- 10.20.193.27:0/714741578 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc8d0062f70 msgr2=0x7fc8d0063410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:39.762 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.758+0000 7fc8d53a56c0 1 --2- 10.20.193.27:0/714741578 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc8d0062f70 0x7fc8d0063410 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:39.762 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.758+0000 7fc8d53a56c0 1 -- 10.20.193.27:0/714741578 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc8d0115440 msgr2=0x7fc8d0062990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:39.763 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.758+0000 7fc8d53a56c0 1 --2- 10.20.193.27:0/714741578 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc8d0115440 0x7fc8d0062990 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:39.763 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.758+0000 7fc8d53a56c0 1 -- 10.20.193.27:0/714741578 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c0009590 con 0x7fc8d010f230 2026-02-13T02:26:39.763 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.758+0000 7fc8d53a56c0 1 --2- 10.20.193.27:0/714741578 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc8d010f230 0x7fc8d0066730 secure :-1 s=READY pgs=481 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc8c002f230 tx=0x7fc8c002fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:39.763 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.758+0000 7fc8be7fc6c0 1 -- 10.20.193.27:0/714741578 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc8c0002ba0 con 0x7fc8d010f230 2026-02-13T02:26:39.763 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.758+0000 7fc8be7fc6c0 1 -- 10.20.193.27:0/714741578 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc8c0002d00 con 0x7fc8d010f230 2026-02-13T02:26:39.763 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.758+0000 7fc8d76306c0 1 -- 10.20.193.27:0/714741578 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8d0168250 con 0x7fc8d010f230 2026-02-13T02:26:39.763 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.758+0000 7fc8be7fc6c0 1 -- 10.20.193.27:0/714741578 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc8c00372f0 con 0x7fc8d010f230 2026-02-13T02:26:39.763 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.758+0000 7fc8d76306c0 1 -- 10.20.193.27:0/714741578 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8d01686d0 con 0x7fc8d010f230 2026-02-13T02:26:39.763 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.759+0000 7fc8d76306c0 1 -- 10.20.193.27:0/714741578 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8d0110500 con 0x7fc8d010f230 2026-02-13T02:26:39.764 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.760+0000 7fc8be7fc6c0 1 -- 10.20.193.27:0/714741578 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7fc8c00078d0 con 0x7fc8d010f230 2026-02-13T02:26:39.764 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.760+0000 7fc8be7fc6c0 1 --2- 10.20.193.27:0/714741578 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc8ac0b5600 0x7fc8ac0d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:39.764 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.760+0000 7fc8be7fc6c0 1 -- 10.20.193.27:0/714741578 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7fc8c0107b90 con 0x7fc8d010f230 2026-02-13T02:26:39.765 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.760+0000 7fc8d4ba46c0 1 --2- 10.20.193.27:0/714741578 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc8ac0b5600 0x7fc8ac0d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:39.765 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.760+0000 7fc8be7fc6c0 1 -- 10.20.193.27:0/714741578 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fc8c00ca6a0 con 0x7fc8d010f230 2026-02-13T02:26:39.765 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.761+0000 7fc8d4ba46c0 1 --2- 10.20.193.27:0/714741578 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc8ac0b5600 0x7fc8ac0d5ab0 secure :-1 s=READY pgs=304 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc8c4005980 tx=0x7fc8c4005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:39.876 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.874+0000 7fc8d76306c0 1 -- 10.20.193.27:0/714741578 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc8d010f600 con 0x7fc8d010f230 2026-02-13T02:26:39.876 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.875+0000 7fc8be7fc6c0 1 -- 10.20.193.27:0/714741578 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7fc8c00cd7c0 con 0x7fc8d010f230 2026-02-13T02:26:39.876 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:26:39.877 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:26:39.878 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.876+0000 7fc8d76306c0 1 -- 10.20.193.27:0/714741578 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc8ac0b5600 msgr2=0x7fc8ac0d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:39.878 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.876+0000 7fc8d76306c0 1 --2- 10.20.193.27:0/714741578 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc8ac0b5600 0x7fc8ac0d5ab0 secure :-1 s=READY pgs=304 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc8c4005980 tx=0x7fc8c4005910 comp rx=0 tx=0).stop 2026-02-13T02:26:39.878 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.876+0000 7fc8d76306c0 1 -- 10.20.193.27:0/714741578 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc8d010f230 msgr2=0x7fc8d0066730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:39.878 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.876+0000 7fc8d76306c0 1 --2- 10.20.193.27:0/714741578 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc8d010f230 0x7fc8d0066730 secure :-1 s=READY pgs=481 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc8c002f230 tx=0x7fc8c002fd40 comp rx=0 tx=0).stop 2026-02-13T02:26:39.878 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.876+0000 7fc8d76306c0 1 -- 10.20.193.27:0/714741578 shutdown_connections 2026-02-13T02:26:39.878 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.876+0000 7fc8d76306c0 1 --2- 10.20.193.27:0/714741578 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fc8ac0b5600 0x7fc8ac0d5ab0 unknown :-1 s=CLOSED pgs=304 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:39.878 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.876+0000 7fc8d76306c0 1 --2- 10.20.193.27:0/714741578 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fc8d0062f70 0x7fc8d0063410 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:39.878 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.876+0000 7fc8d76306c0 1 --2- 10.20.193.27:0/714741578 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc8d0115440 0x7fc8d0062990 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:39.878 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.876+0000 7fc8d76306c0 1 --2- 10.20.193.27:0/714741578 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fc8d010f230 0x7fc8d0066730 unknown :-1 s=CLOSED pgs=481 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:39.878 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.876+0000 7fc8d76306c0 1 -- 10.20.193.27:0/714741578 >> 10.20.193.27:0/714741578 conn(0x7fc8d00ecb30 msgr2=0x7fc8d010cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:39.879 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.876+0000 7fc8d76306c0 1 -- 10.20.193.27:0/714741578 shutdown_connections 2026-02-13T02:26:39.879 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:39.876+0000 7fc8d76306c0 1 -- 10.20.193.27:0/714741578 wait complete. 2026-02-13T02:26:40.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:40 trial028 ceph-mon[26701]: pgmap v322: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:26:40.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:40 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/714741578' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:40.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:40 trial107 ceph-mon[27422]: pgmap v322: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:26:40.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:40 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/714741578' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:40.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:40 trial027 ceph-mon[24567]: pgmap v322: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:26:40.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:40 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/714741578' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:42.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:42 trial028 ceph-mon[26701]: pgmap v323: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:42.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:42 trial107 ceph-mon[27422]: pgmap v323: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:42.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:42 trial027 ceph-mon[24567]: pgmap v323: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:42.919 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:26:43.039 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:26:43.168 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.163+0000 7f0c7d28c6c0 1 -- 10.20.193.27:0/3824544627 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0c78115410 msgr2=0x7f0c78175db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:43.168 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.163+0000 7f0c7d28c6c0 1 --2- 10.20.193.27:0/3824544627 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0c78115410 0x7f0c78175db0 secure :-1 s=READY pgs=482 gs=1 cs=0 l=1 c_cookie=10903538692014577786 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0c6800aa70 tx=0x7f0c6802fde0 comp rx=0 tx=0).stop 2026-02-13T02:26:43.168 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.163+0000 7f0c7d28c6c0 1 -- 10.20.193.27:0/3824544627 shutdown_connections 2026-02-13T02:26:43.168 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.163+0000 7f0c7d28c6c0 1 --2- 10.20.193.27:0/3824544627 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0c78115410 0x7f0c78175db0 unknown :-1 s=CLOSED pgs=482 gs=1 cs=0 l=1 c_cookie=10903538692014577786 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:43.168 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.163+0000 7f0c7d28c6c0 1 --2- 10.20.193.27:0/3824544627 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0c7810fbf0 0x7f0c78114c30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:43.168 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.163+0000 7f0c7d28c6c0 1 --2- 10.20.193.27:0/3824544627 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f0c7810f200 0x7f0c7810f5d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:43.168 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.163+0000 7f0c7d28c6c0 1 -- 10.20.193.27:0/3824544627 >> 10.20.193.27:0/3824544627 conn(0x7f0c780ecb00 msgr2=0x7f0c7810cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:43.168 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.163+0000 7f0c7d28c6c0 1 -- 10.20.193.27:0/3824544627 shutdown_connections 2026-02-13T02:26:43.168 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.163+0000 7f0c7d28c6c0 1 -- 10.20.193.27:0/3824544627 wait complete. 2026-02-13T02:26:43.169 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.163+0000 7f0c7d28c6c0 1 Processor -- start 2026-02-13T02:26:43.169 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.163+0000 7f0c7d28c6c0 1 -- start start 2026-02-13T02:26:43.169 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.163+0000 7f0c7d28c6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f0c7810f200 0x7f0c7811d290 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:43.169 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.163+0000 7f0c7d28c6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0c78115410 0x7f0c7811d7d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:43.169 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.163+0000 7f0c7d28c6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0c78122b90 0x7f0c78123030 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:43.169 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.163+0000 7f0c7d28c6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0c781235a0 con 0x7f0c78115410 2026-02-13T02:26:43.169 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.163+0000 7f0c7d28c6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0c78123710 con 0x7f0c78122b90 2026-02-13T02:26:43.169 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.163+0000 7f0c7d28c6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0c781238d0 con 0x7f0c7810f200 2026-02-13T02:26:43.169 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.164+0000 7f0c767fc6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0c78115410 0x7f0c7811d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:43.169 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.164+0000 7f0c767fc6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0c78115410 0x7f0c7811d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55608/0 (socket says 10.20.193.27:55608) 2026-02-13T02:26:43.170 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.164+0000 7f0c767fc6c0 1 -- 10.20.193.27:0/142494799 learned_addr learned my addr 10.20.193.27:0/142494799 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:26:43.170 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.164+0000 7f0c777fe6c0 1 --2- 10.20.193.27:0/142494799 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0c78122b90 0x7f0c78123030 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:43.170 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.164+0000 7f0c76ffd6c0 1 --2- 10.20.193.27:0/142494799 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f0c7810f200 0x7f0c7811d290 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:43.170 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.164+0000 7f0c767fc6c0 1 -- 10.20.193.27:0/142494799 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f0c7810f200 msgr2=0x7f0c7811d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:43.170 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.164+0000 7f0c767fc6c0 1 --2- 10.20.193.27:0/142494799 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f0c7810f200 0x7f0c7811d290 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:43.170 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.164+0000 7f0c767fc6c0 1 -- 10.20.193.27:0/142494799 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0c78122b90 msgr2=0x7f0c78123030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:43.170 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.164+0000 7f0c767fc6c0 1 --2- 10.20.193.27:0/142494799 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0c78122b90 0x7f0c78123030 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:43.170 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.164+0000 7f0c767fc6c0 1 -- 10.20.193.27:0/142494799 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c6800a660 con 0x7f0c78115410 2026-02-13T02:26:43.170 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.164+0000 7f0c76ffd6c0 1 --2- 10.20.193.27:0/142494799 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f0c7810f200 0x7f0c7811d290 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:26:43.171 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.164+0000 7f0c767fc6c0 1 --2- 10.20.193.27:0/142494799 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0c78115410 0x7f0c7811d7d0 secure :-1 s=READY pgs=483 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0c60005dc0 tx=0x7f0c6000bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:43.171 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.164+0000 7f0c57fff6c0 1 -- 10.20.193.27:0/142494799 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0c60004250 con 0x7f0c78115410 2026-02-13T02:26:43.171 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.164+0000 7f0c57fff6c0 1 -- 10.20.193.27:0/142494799 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f0c60009070 con 0x7f0c78115410 2026-02-13T02:26:43.171 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.164+0000 7f0c57fff6c0 1 -- 10.20.193.27:0/142494799 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0c6000cd50 con 0x7f0c78115410 2026-02-13T02:26:43.171 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.164+0000 7f0c7d28c6c0 1 -- 10.20.193.27:0/142494799 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c78163d80 con 0x7f0c78115410 2026-02-13T02:26:43.171 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.164+0000 7f0c7d28c6c0 1 -- 10.20.193.27:0/142494799 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c78164260 con 0x7f0c78115410 2026-02-13T02:26:43.171 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.164+0000 7f0c7d28c6c0 1 -- 10.20.193.27:0/142494799 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c781104d0 con 0x7f0c78115410 2026-02-13T02:26:43.171 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.166+0000 7f0c57fff6c0 1 -- 10.20.193.27:0/142494799 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f0c600028c0 con 0x7f0c78115410 2026-02-13T02:26:43.172 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.166+0000 7f0c57fff6c0 1 --2- 10.20.193.27:0/142494799 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f0c4c0b5680 0x7f0c4c0d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:43.172 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.166+0000 7f0c76ffd6c0 1 --2- 10.20.193.27:0/142494799 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f0c4c0b5680 0x7f0c4c0d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:43.173 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.167+0000 7f0c57fff6c0 1 -- 10.20.193.27:0/142494799 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f0c600e39f0 con 0x7f0c78115410 2026-02-13T02:26:43.173 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.167+0000 7f0c76ffd6c0 1 --2- 10.20.193.27:0/142494799 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f0c4c0b5680 0x7f0c4c0d5b30 secure :-1 s=READY pgs=305 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0c6c004750 tx=0x7f0c6c005f10 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:43.173 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.167+0000 7f0c57fff6c0 1 -- 10.20.193.27:0/142494799 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f0c600a9ed0 con 0x7f0c78115410 2026-02-13T02:26:43.284 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.283+0000 7f0c7d28c6c0 1 -- 10.20.193.27:0/142494799 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0c7811e3e0 con 0x7f0c78115410 2026-02-13T02:26:43.284 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.283+0000 7f0c57fff6c0 1 -- 10.20.193.27:0/142494799 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f0c600704a0 con 0x7f0c78115410 2026-02-13T02:26:43.285 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:26:43.285 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:26:43.286 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.284+0000 7f0c7d28c6c0 1 -- 10.20.193.27:0/142494799 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f0c4c0b5680 msgr2=0x7f0c4c0d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:43.286 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.284+0000 7f0c7d28c6c0 1 --2- 10.20.193.27:0/142494799 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f0c4c0b5680 0x7f0c4c0d5b30 secure :-1 s=READY pgs=305 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0c6c004750 tx=0x7f0c6c005f10 comp rx=0 tx=0).stop 2026-02-13T02:26:43.286 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.284+0000 7f0c7d28c6c0 1 -- 10.20.193.27:0/142494799 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0c78115410 msgr2=0x7f0c7811d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:43.286 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.284+0000 7f0c7d28c6c0 1 --2- 10.20.193.27:0/142494799 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0c78115410 0x7f0c7811d7d0 secure :-1 s=READY pgs=483 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0c60005dc0 tx=0x7f0c6000bbb0 comp rx=0 tx=0).stop 2026-02-13T02:26:43.286 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.284+0000 7f0c7d28c6c0 1 -- 10.20.193.27:0/142494799 shutdown_connections 2026-02-13T02:26:43.286 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.284+0000 7f0c7d28c6c0 1 --2- 10.20.193.27:0/142494799 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f0c4c0b5680 0x7f0c4c0d5b30 unknown :-1 s=CLOSED pgs=305 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:43.286 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.284+0000 7f0c7d28c6c0 1 --2- 10.20.193.27:0/142494799 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0c78122b90 0x7f0c78123030 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:43.286 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.284+0000 7f0c7d28c6c0 1 --2- 10.20.193.27:0/142494799 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0c78115410 0x7f0c7811d7d0 unknown :-1 s=CLOSED pgs=483 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:43.286 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.284+0000 7f0c7d28c6c0 1 --2- 10.20.193.27:0/142494799 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f0c7810f200 0x7f0c7811d290 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:43.286 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.284+0000 7f0c7d28c6c0 1 -- 10.20.193.27:0/142494799 >> 10.20.193.27:0/142494799 conn(0x7f0c780ecb00 msgr2=0x7f0c7810cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:43.287 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.284+0000 7f0c7d28c6c0 1 -- 10.20.193.27:0/142494799 shutdown_connections 2026-02-13T02:26:43.287 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:43.284+0000 7f0c7d28c6c0 1 -- 10.20.193.27:0/142494799 wait complete. 2026-02-13T02:26:44.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:44 trial028 ceph-mon[26701]: pgmap v324: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:26:44.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:44 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/142494799' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:44.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:44 trial107 ceph-mon[27422]: pgmap v324: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:26:44.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:44 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/142494799' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:44.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:44 trial027 ceph-mon[24567]: pgmap v324: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:26:44.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:44 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/142494799' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:46.328 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:26:46.446 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:26:46.468 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:46 trial027 ceph-mon[24567]: pgmap v325: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:46.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:46 trial028 ceph-mon[26701]: pgmap v325: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:46.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:46 trial107 ceph-mon[27422]: pgmap v325: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:46.561 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.557+0000 7fb14b81c6c0 1 -- 10.20.193.27:0/2533113288 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb14410f200 msgr2=0x7fb14410f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:46.561 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.557+0000 7fb14b81c6c0 1 --2- 10.20.193.27:0/2533113288 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb14410f200 0x7fb14410f5d0 secure :-1 s=READY pgs=60 gs=2 cs=0 l=1 c_cookie=15245608118069303385 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb134009970 tx=0x7fb13402ed20 comp rx=0 tx=0).stop 2026-02-13T02:26:46.561 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.557+0000 7fb14b81c6c0 1 -- 10.20.193.27:0/2533113288 shutdown_connections 2026-02-13T02:26:46.561 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.557+0000 7fb14b81c6c0 1 --2- 10.20.193.27:0/2533113288 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb144115410 0x7fb144175db0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:46.561 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.557+0000 7fb14b81c6c0 1 --2- 10.20.193.27:0/2533113288 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb14410fbf0 0x7fb144114c30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:46.561 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.557+0000 7fb14b81c6c0 1 --2- 10.20.193.27:0/2533113288 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb14410f200 0x7fb14410f5d0 unknown :-1 s=CLOSED pgs=60 gs=2 cs=0 l=1 c_cookie=15245608118069303385 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:46.561 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.557+0000 7fb14b81c6c0 1 -- 10.20.193.27:0/2533113288 >> 10.20.193.27:0/2533113288 conn(0x7fb1440ecb00 msgr2=0x7fb14410cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:46.562 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.557+0000 7fb14b81c6c0 1 -- 10.20.193.27:0/2533113288 shutdown_connections 2026-02-13T02:26:46.562 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.557+0000 7fb14b81c6c0 1 -- 10.20.193.27:0/2533113288 wait complete. 2026-02-13T02:26:46.562 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.558+0000 7fb14b81c6c0 1 Processor -- start 2026-02-13T02:26:46.562 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.558+0000 7fb14b81c6c0 1 -- start start 2026-02-13T02:26:46.562 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.558+0000 7fb14b81c6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb14410f200 0x7fb14411d250 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:46.562 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.558+0000 7fb14b81c6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb144115410 0x7fb14411d790 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:46.562 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.558+0000 7fb14b81c6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb144122b50 0x7fb144122ff0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:46.562 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.558+0000 7fb14b81c6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb144123560 con 0x7fb14410f200 2026-02-13T02:26:46.562 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.558+0000 7fb14b81c6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb1441236d0 con 0x7fb144122b50 2026-02-13T02:26:46.562 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.558+0000 7fb14b81c6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb144123890 con 0x7fb144115410 2026-02-13T02:26:46.562 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.558+0000 7fb14958d6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb14410f200 0x7fb14411d250 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:46.562 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.558+0000 7fb14958d6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb14410f200 0x7fb14411d250 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:55622/0 (socket says 10.20.193.27:55622) 2026-02-13T02:26:46.563 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.558+0000 7fb14958d6c0 1 -- 10.20.193.27:0/2798628962 learned_addr learned my addr 10.20.193.27:0/2798628962 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:26:46.563 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.558+0000 7fb149d8e6c0 1 --2- 10.20.193.27:0/2798628962 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb144122b50 0x7fb144122ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:46.563 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.558+0000 7fb148d8c6c0 1 --2- 10.20.193.27:0/2798628962 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb144115410 0x7fb14411d790 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:46.563 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.558+0000 7fb14958d6c0 1 -- 10.20.193.27:0/2798628962 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb144115410 msgr2=0x7fb14411d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:46.563 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.558+0000 7fb14958d6c0 1 --2- 10.20.193.27:0/2798628962 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb144115410 0x7fb14411d790 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:46.563 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.558+0000 7fb14958d6c0 1 -- 10.20.193.27:0/2798628962 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb144122b50 msgr2=0x7fb144122ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:46.563 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.558+0000 7fb14958d6c0 1 --2- 10.20.193.27:0/2798628962 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb144122b50 0x7fb144122ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:46.563 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.558+0000 7fb14958d6c0 1 -- 10.20.193.27:0/2798628962 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14000a5b0 con 0x7fb14410f200 2026-02-13T02:26:46.563 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.558+0000 7fb14958d6c0 1 --2- 10.20.193.27:0/2798628962 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb14410f200 0x7fb14411d250 secure :-1 s=READY pgs=484 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb13402fc30 tx=0x7fb13402fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:46.563 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.558+0000 7fb1327fc6c0 1 -- 10.20.193.27:0/2798628962 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb134002ae0 con 0x7fb14410f200 2026-02-13T02:26:46.563 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.558+0000 7fb14b81c6c0 1 -- 10.20.193.27:0/2798628962 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb134009590 con 0x7fb14410f200 2026-02-13T02:26:46.563 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.558+0000 7fb1327fc6c0 1 -- 10.20.193.27:0/2798628962 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb134002c40 con 0x7fb14410f200 2026-02-13T02:26:46.564 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.558+0000 7fb1327fc6c0 1 -- 10.20.193.27:0/2798628962 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb1340372f0 con 0x7fb14410f200 2026-02-13T02:26:46.564 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.558+0000 7fb14b81c6c0 1 -- 10.20.193.27:0/2798628962 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb144163fd0 con 0x7fb14410f200 2026-02-13T02:26:46.564 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.559+0000 7fb14b81c6c0 1 -- 10.20.193.27:0/2798628962 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1441104d0 con 0x7fb14410f200 2026-02-13T02:26:46.565 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.560+0000 7fb1327fc6c0 1 -- 10.20.193.27:0/2798628962 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7fb1340071c0 con 0x7fb14410f200 2026-02-13T02:26:46.565 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.561+0000 7fb1327fc6c0 1 --2- 10.20.193.27:0/2798628962 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb1200b5600 0x7fb1200d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:46.565 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.561+0000 7fb148d8c6c0 1 --2- 10.20.193.27:0/2798628962 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb1200b5600 0x7fb1200d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:46.565 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.561+0000 7fb1327fc6c0 1 -- 10.20.193.27:0/2798628962 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7fb134107cf0 con 0x7fb14410f200 2026-02-13T02:26:46.565 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.562+0000 7fb1327fc6c0 1 -- 10.20.193.27:0/2798628962 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fb1340ca770 con 0x7fb14410f200 2026-02-13T02:26:46.565 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.562+0000 7fb148d8c6c0 1 --2- 10.20.193.27:0/2798628962 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb1200b5600 0x7fb1200d5ab0 secure :-1 s=READY pgs=306 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb144123be0 tx=0x7fb138009260 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:46.677 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.676+0000 7fb14b81c6c0 1 -- 10.20.193.27:0/2798628962 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb14410f5d0 con 0x7fb14410f200 2026-02-13T02:26:46.678 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.676+0000 7fb1327fc6c0 1 -- 10.20.193.27:0/2798628962 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7fb1340cd920 con 0x7fb14410f200 2026-02-13T02:26:46.678 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:26:46.678 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:26:46.679 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.677+0000 7fb14b81c6c0 1 -- 10.20.193.27:0/2798628962 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb1200b5600 msgr2=0x7fb1200d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:46.679 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.677+0000 7fb14b81c6c0 1 --2- 10.20.193.27:0/2798628962 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb1200b5600 0x7fb1200d5ab0 secure :-1 s=READY pgs=306 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb144123be0 tx=0x7fb138009260 comp rx=0 tx=0).stop 2026-02-13T02:26:46.679 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.677+0000 7fb14b81c6c0 1 -- 10.20.193.27:0/2798628962 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb14410f200 msgr2=0x7fb14411d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:46.679 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.677+0000 7fb14b81c6c0 1 --2- 10.20.193.27:0/2798628962 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb14410f200 0x7fb14411d250 secure :-1 s=READY pgs=484 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb13402fc30 tx=0x7fb13402fc60 comp rx=0 tx=0).stop 2026-02-13T02:26:46.679 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.677+0000 7fb14b81c6c0 1 -- 10.20.193.27:0/2798628962 shutdown_connections 2026-02-13T02:26:46.680 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.677+0000 7fb14b81c6c0 1 --2- 10.20.193.27:0/2798628962 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fb1200b5600 0x7fb1200d5ab0 unknown :-1 s=CLOSED pgs=306 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:46.680 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.677+0000 7fb14b81c6c0 1 --2- 10.20.193.27:0/2798628962 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb144122b50 0x7fb144122ff0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:46.680 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.677+0000 7fb14b81c6c0 1 --2- 10.20.193.27:0/2798628962 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fb144115410 0x7fb14411d790 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:46.680 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.677+0000 7fb14b81c6c0 1 --2- 10.20.193.27:0/2798628962 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fb14410f200 0x7fb14411d250 unknown :-1 s=CLOSED pgs=484 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:46.680 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.677+0000 7fb14b81c6c0 1 -- 10.20.193.27:0/2798628962 >> 10.20.193.27:0/2798628962 conn(0x7fb1440ecb00 msgr2=0x7fb14410cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:46.680 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.677+0000 7fb14b81c6c0 1 -- 10.20.193.27:0/2798628962 shutdown_connections 2026-02-13T02:26:46.680 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:46.677+0000 7fb14b81c6c0 1 -- 10.20.193.27:0/2798628962 wait complete. 2026-02-13T02:26:47.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:47 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2798628962' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:47.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:47 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2798628962' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:47.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:47 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2798628962' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:48.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:48 trial028 ceph-mon[26701]: pgmap v326: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:48.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:48 trial107 ceph-mon[27422]: pgmap v326: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:48.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:48 trial027 ceph-mon[24567]: pgmap v326: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:49.719 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:26:49.835 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:26:49.950 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.946+0000 7f3b2efba6c0 1 -- 10.20.193.27:0/382643771 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3b28115260 msgr2=0x7f3b28175c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:49.950 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.946+0000 7f3b2efba6c0 1 --2- 10.20.193.27:0/382643771 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3b28115260 0x7f3b28175c00 secure :-1 s=READY pgs=485 gs=1 cs=0 l=1 c_cookie=15114279600478450133 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3b2400aa70 tx=0x7f3b2402fde0 comp rx=0 tx=0).stop 2026-02-13T02:26:49.950 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.946+0000 7f3b2efba6c0 1 -- 10.20.193.27:0/382643771 shutdown_connections 2026-02-13T02:26:49.950 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.946+0000 7f3b2efba6c0 1 --2- 10.20.193.27:0/382643771 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3b28115260 0x7f3b28175c00 unknown :-1 s=CLOSED pgs=485 gs=1 cs=0 l=1 c_cookie=15114279600478450133 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:49.950 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.946+0000 7f3b2efba6c0 1 --2- 10.20.193.27:0/382643771 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3b2810fa40 0x7f3b28114a80 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:49.950 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.946+0000 7f3b2efba6c0 1 --2- 10.20.193.27:0/382643771 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3b2810f050 0x7f3b2810f420 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:49.951 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.946+0000 7f3b2efba6c0 1 -- 10.20.193.27:0/382643771 >> 10.20.193.27:0/382643771 conn(0x7f3b280ec950 msgr2=0x7f3b2810cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:49.951 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.946+0000 7f3b2efba6c0 1 -- 10.20.193.27:0/382643771 shutdown_connections 2026-02-13T02:26:49.951 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.946+0000 7f3b2efba6c0 1 -- 10.20.193.27:0/382643771 wait complete. 2026-02-13T02:26:49.951 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.946+0000 7f3b2efba6c0 1 Processor -- start 2026-02-13T02:26:49.951 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.946+0000 7f3b2efba6c0 1 -- start start 2026-02-13T02:26:49.951 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.946+0000 7f3b2efba6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3b2810f050 0x7f3b2811d060 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:49.951 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.946+0000 7f3b2efba6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3b28115260 0x7f3b2811d5a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:49.951 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.946+0000 7f3b2efba6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3b28122960 0x7f3b28122e00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:49.951 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.946+0000 7f3b2efba6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3b28123370 con 0x7f3b28122960 2026-02-13T02:26:49.951 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.946+0000 7f3b2efba6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3b281234e0 con 0x7f3b2810f050 2026-02-13T02:26:49.951 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.946+0000 7f3b2efba6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3b281236a0 con 0x7f3b28115260 2026-02-13T02:26:49.952 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.946+0000 7f3b2e7b96c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3b28122960 0x7f3b28122e00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:49.952 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.946+0000 7f3b2e7b96c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3b28122960 0x7f3b28122e00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:48594/0 (socket says 10.20.193.27:48594) 2026-02-13T02:26:49.952 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.946+0000 7f3b2e7b96c0 1 -- 10.20.193.27:0/775599943 learned_addr learned my addr 10.20.193.27:0/775599943 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:26:49.952 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.946+0000 7f3b2dfb86c0 1 --2- 10.20.193.27:0/775599943 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3b2810f050 0x7f3b2811d060 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:49.952 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.946+0000 7f3b2e7b96c0 1 -- 10.20.193.27:0/775599943 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3b28115260 msgr2=0x7f3b2811d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:49.952 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.946+0000 7f3b2e7b96c0 1 --2- 10.20.193.27:0/775599943 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3b28115260 0x7f3b2811d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:49.952 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.946+0000 7f3b2e7b96c0 1 -- 10.20.193.27:0/775599943 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3b2810f050 msgr2=0x7f3b2811d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:49.952 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.946+0000 7f3b2e7b96c0 1 --2- 10.20.193.27:0/775599943 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3b2810f050 0x7f3b2811d060 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:49.952 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.946+0000 7f3b2e7b96c0 1 -- 10.20.193.27:0/775599943 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b2400a660 con 0x7f3b28122960 2026-02-13T02:26:49.952 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.946+0000 7f3b2dfb86c0 1 --2- 10.20.193.27:0/775599943 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3b2810f050 0x7f3b2811d060 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:26:49.952 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.947+0000 7f3b2e7b96c0 1 --2- 10.20.193.27:0/775599943 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3b28122960 0x7f3b28122e00 secure :-1 s=READY pgs=486 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3b240054b0 tx=0x7f3b240054e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:49.952 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.947+0000 7f3b16ffd6c0 1 -- 10.20.193.27:0/775599943 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3b24003a30 con 0x7f3b28122960 2026-02-13T02:26:49.953 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.947+0000 7f3b2efba6c0 1 -- 10.20.193.27:0/775599943 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b28163af0 con 0x7f3b28122960 2026-02-13T02:26:49.953 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.947+0000 7f3b16ffd6c0 1 -- 10.20.193.27:0/775599943 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3b24003b90 con 0x7f3b28122960 2026-02-13T02:26:49.953 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.947+0000 7f3b16ffd6c0 1 -- 10.20.193.27:0/775599943 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3b240382f0 con 0x7f3b28122960 2026-02-13T02:26:49.953 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.947+0000 7f3b2efba6c0 1 -- 10.20.193.27:0/775599943 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b28163f20 con 0x7f3b28122960 2026-02-13T02:26:49.953 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.947+0000 7f3b2efba6c0 1 -- 10.20.193.27:0/775599943 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b28110320 con 0x7f3b28122960 2026-02-13T02:26:49.954 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.948+0000 7f3b16ffd6c0 1 -- 10.20.193.27:0/775599943 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f3b24008260 con 0x7f3b28122960 2026-02-13T02:26:49.954 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.949+0000 7f3b16ffd6c0 1 --2- 10.20.193.27:0/775599943 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3b040b5680 0x7f3b040d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:49.954 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.949+0000 7f3b2dfb86c0 1 --2- 10.20.193.27:0/775599943 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3b040b5680 0x7f3b040d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:49.954 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.949+0000 7f3b16ffd6c0 1 -- 10.20.193.27:0/775599943 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f3b24108e40 con 0x7f3b28122960 2026-02-13T02:26:49.954 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.949+0000 7f3b16ffd6c0 1 -- 10.20.193.27:0/775599943 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f3b24109330 con 0x7f3b28122960 2026-02-13T02:26:49.954 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:49.949+0000 7f3b2dfb86c0 1 --2- 10.20.193.27:0/775599943 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3b040b5680 0x7f3b040d5b30 secure :-1 s=READY pgs=307 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3b18004750 tx=0x7f3b18005f10 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:50.064 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:50.062+0000 7f3b2efba6c0 1 -- 10.20.193.27:0/775599943 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3b2810fa40 con 0x7f3b28122960 2026-02-13T02:26:50.064 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:50.062+0000 7f3b16ffd6c0 1 -- 10.20.193.27:0/775599943 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f3b240cb900 con 0x7f3b28122960 2026-02-13T02:26:50.065 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:26:50.065 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:26:50.066 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:50.063+0000 7f3b2efba6c0 1 -- 10.20.193.27:0/775599943 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3b040b5680 msgr2=0x7f3b040d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:50.066 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:50.063+0000 7f3b2efba6c0 1 --2- 10.20.193.27:0/775599943 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3b040b5680 0x7f3b040d5b30 secure :-1 s=READY pgs=307 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3b18004750 tx=0x7f3b18005f10 comp rx=0 tx=0).stop 2026-02-13T02:26:50.066 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:50.063+0000 7f3b2efba6c0 1 -- 10.20.193.27:0/775599943 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3b28122960 msgr2=0x7f3b28122e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:50.066 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:50.063+0000 7f3b2efba6c0 1 --2- 10.20.193.27:0/775599943 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3b28122960 0x7f3b28122e00 secure :-1 s=READY pgs=486 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3b240054b0 tx=0x7f3b240054e0 comp rx=0 tx=0).stop 2026-02-13T02:26:50.066 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:50.063+0000 7f3b2efba6c0 1 -- 10.20.193.27:0/775599943 shutdown_connections 2026-02-13T02:26:50.066 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:50.063+0000 7f3b2efba6c0 1 --2- 10.20.193.27:0/775599943 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f3b040b5680 0x7f3b040d5b30 unknown :-1 s=CLOSED pgs=307 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:50.066 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:50.063+0000 7f3b2efba6c0 1 --2- 10.20.193.27:0/775599943 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f3b28122960 0x7f3b28122e00 unknown :-1 s=CLOSED pgs=486 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:50.066 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:50.063+0000 7f3b2efba6c0 1 --2- 10.20.193.27:0/775599943 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f3b28115260 0x7f3b2811d5a0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:50.067 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:50.063+0000 7f3b2efba6c0 1 --2- 10.20.193.27:0/775599943 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3b2810f050 0x7f3b2811d060 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:50.067 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:50.063+0000 7f3b2efba6c0 1 -- 10.20.193.27:0/775599943 >> 10.20.193.27:0/775599943 conn(0x7f3b280ec950 msgr2=0x7f3b2810cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:50.067 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:50.063+0000 7f3b2efba6c0 1 -- 10.20.193.27:0/775599943 shutdown_connections 2026-02-13T02:26:50.067 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:50.063+0000 7f3b2efba6c0 1 -- 10.20.193.27:0/775599943 wait complete. 2026-02-13T02:26:50.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:50 trial027 ceph-mon[24567]: pgmap v327: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:26:50.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:50 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/775599943' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:50.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:50 trial028 ceph-mon[26701]: pgmap v327: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:26:50.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:50 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/775599943' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:50.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:50 trial107 ceph-mon[27422]: pgmap v327: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:26:50.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:50 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/775599943' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:52.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:52 trial028 ceph-mon[26701]: pgmap v328: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:52.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:52 trial107 ceph-mon[27422]: pgmap v328: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:52.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:52 trial027 ceph-mon[24567]: pgmap v328: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:53.107 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:26:53.226 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:26:53.340 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.336+0000 7ff1255f86c0 1 -- 10.20.193.27:0/1677220491 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff120115440 msgr2=0x7ff120175de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:53.341 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.336+0000 7ff1255f86c0 1 --2- 10.20.193.27:0/1677220491 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff120115440 0x7ff120175de0 secure :-1 s=READY pgs=487 gs=1 cs=0 l=1 c_cookie=9352537579104381549 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff11000aa70 tx=0x7ff11002fde0 comp rx=0 tx=0).stop 2026-02-13T02:26:53.341 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.336+0000 7ff1255f86c0 1 -- 10.20.193.27:0/1677220491 shutdown_connections 2026-02-13T02:26:53.341 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.336+0000 7ff1255f86c0 1 --2- 10.20.193.27:0/1677220491 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff120115440 0x7ff120175de0 unknown :-1 s=CLOSED pgs=487 gs=1 cs=0 l=1 c_cookie=9352537579104381549 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:53.341 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.336+0000 7ff1255f86c0 1 --2- 10.20.193.27:0/1677220491 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff12010fc20 0x7ff120114c60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:53.341 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.336+0000 7ff1255f86c0 1 --2- 10.20.193.27:0/1677220491 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff12010f230 0x7ff12010f600 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:53.341 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.336+0000 7ff1255f86c0 1 -- 10.20.193.27:0/1677220491 >> 10.20.193.27:0/1677220491 conn(0x7ff1200ecb30 msgr2=0x7ff12010cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:53.341 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.336+0000 7ff1255f86c0 1 -- 10.20.193.27:0/1677220491 shutdown_connections 2026-02-13T02:26:53.341 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.336+0000 7ff1255f86c0 1 -- 10.20.193.27:0/1677220491 wait complete. 2026-02-13T02:26:53.341 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.336+0000 7ff1255f86c0 1 Processor -- start 2026-02-13T02:26:53.341 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.336+0000 7ff1255f86c0 1 -- start start 2026-02-13T02:26:53.342 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.336+0000 7ff1255f86c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff12010f230 0x7ff12011d1e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:53.342 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.336+0000 7ff1255f86c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff120115440 0x7ff12011d720 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:53.342 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.336+0000 7ff1255f86c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff120122ae0 0x7ff120122f80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:53.342 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.336+0000 7ff1255f86c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff1201234f0 con 0x7ff120122ae0 2026-02-13T02:26:53.342 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.336+0000 7ff1255f86c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff120123660 con 0x7ff12010f230 2026-02-13T02:26:53.342 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.336+0000 7ff1255f86c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff120123820 con 0x7ff120115440 2026-02-13T02:26:53.342 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.337+0000 7ff11f7fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff120122ae0 0x7ff120122f80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:53.342 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.337+0000 7ff11f7fe6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff120122ae0 0x7ff120122f80 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:48616/0 (socket says 10.20.193.27:48616) 2026-02-13T02:26:53.342 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.337+0000 7ff11f7fe6c0 1 -- 10.20.193.27:0/4210216639 learned_addr learned my addr 10.20.193.27:0/4210216639 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:26:53.342 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.337+0000 7ff11effd6c0 1 --2- 10.20.193.27:0/4210216639 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff12010f230 0x7ff12011d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:53.342 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.337+0000 7ff11f7fe6c0 1 -- 10.20.193.27:0/4210216639 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff120115440 msgr2=0x7ff12011d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:53.343 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.337+0000 7ff11f7fe6c0 1 --2- 10.20.193.27:0/4210216639 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff120115440 0x7ff12011d720 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:53.343 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.337+0000 7ff11f7fe6c0 1 -- 10.20.193.27:0/4210216639 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff12010f230 msgr2=0x7ff12011d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:53.343 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.337+0000 7ff11f7fe6c0 1 --2- 10.20.193.27:0/4210216639 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff12010f230 0x7ff12011d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:53.343 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.337+0000 7ff11f7fe6c0 1 -- 10.20.193.27:0/4210216639 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff11000a660 con 0x7ff120122ae0 2026-02-13T02:26:53.343 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.337+0000 7ff11effd6c0 1 --2- 10.20.193.27:0/4210216639 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff12010f230 0x7ff12011d1e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:26:53.343 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.337+0000 7ff11f7fe6c0 1 --2- 10.20.193.27:0/4210216639 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff120122ae0 0x7ff120122f80 secure :-1 s=READY pgs=488 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff1100054b0 tx=0x7ff1100054e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:53.343 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.337+0000 7ff0fffff6c0 1 -- 10.20.193.27:0/4210216639 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff110003a30 con 0x7ff120122ae0 2026-02-13T02:26:53.343 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.337+0000 7ff0fffff6c0 1 -- 10.20.193.27:0/4210216639 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff110003b90 con 0x7ff120122ae0 2026-02-13T02:26:53.343 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.337+0000 7ff1255f86c0 1 -- 10.20.193.27:0/4210216639 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff120163c70 con 0x7ff120122ae0 2026-02-13T02:26:53.343 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.337+0000 7ff0fffff6c0 1 -- 10.20.193.27:0/4210216639 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff1100382f0 con 0x7ff120122ae0 2026-02-13T02:26:53.343 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.337+0000 7ff1255f86c0 1 -- 10.20.193.27:0/4210216639 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1201640a0 con 0x7ff120122ae0 2026-02-13T02:26:53.343 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.337+0000 7ff1255f86c0 1 -- 10.20.193.27:0/4210216639 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff120110500 con 0x7ff120122ae0 2026-02-13T02:26:53.344 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.339+0000 7ff0fffff6c0 1 -- 10.20.193.27:0/4210216639 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7ff110008260 con 0x7ff120122ae0 2026-02-13T02:26:53.345 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.340+0000 7ff0fffff6c0 1 --2- 10.20.193.27:0/4210216639 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff0f40b5680 0x7ff0f40d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:53.345 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.340+0000 7ff11effd6c0 1 --2- 10.20.193.27:0/4210216639 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff0f40b5680 0x7ff0f40d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:53.345 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.340+0000 7ff0fffff6c0 1 -- 10.20.193.27:0/4210216639 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7ff110108dc0 con 0x7ff120122ae0 2026-02-13T02:26:53.345 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.340+0000 7ff0fffff6c0 1 -- 10.20.193.27:0/4210216639 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7ff110109240 con 0x7ff120122ae0 2026-02-13T02:26:53.345 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.340+0000 7ff11effd6c0 1 --2- 10.20.193.27:0/4210216639 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff0f40b5680 0x7ff0f40d5b30 secure :-1 s=READY pgs=308 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff114004750 tx=0x7ff114005f10 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:53.458 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.456+0000 7ff1255f86c0 1 -- 10.20.193.27:0/4210216639 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff12010f600 con 0x7ff120122ae0 2026-02-13T02:26:53.458 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.456+0000 7ff0fffff6c0 1 -- 10.20.193.27:0/4210216639 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7ff1100cb840 con 0x7ff120122ae0 2026-02-13T02:26:53.459 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:26:53.459 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:26:53.460 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.457+0000 7ff1255f86c0 1 -- 10.20.193.27:0/4210216639 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff0f40b5680 msgr2=0x7ff0f40d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:53.460 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.457+0000 7ff1255f86c0 1 --2- 10.20.193.27:0/4210216639 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff0f40b5680 0x7ff0f40d5b30 secure :-1 s=READY pgs=308 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff114004750 tx=0x7ff114005f10 comp rx=0 tx=0).stop 2026-02-13T02:26:53.460 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.457+0000 7ff1255f86c0 1 -- 10.20.193.27:0/4210216639 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff120122ae0 msgr2=0x7ff120122f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:53.460 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.457+0000 7ff1255f86c0 1 --2- 10.20.193.27:0/4210216639 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff120122ae0 0x7ff120122f80 secure :-1 s=READY pgs=488 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff1100054b0 tx=0x7ff1100054e0 comp rx=0 tx=0).stop 2026-02-13T02:26:53.460 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.457+0000 7ff1255f86c0 1 -- 10.20.193.27:0/4210216639 shutdown_connections 2026-02-13T02:26:53.460 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.457+0000 7ff1255f86c0 1 --2- 10.20.193.27:0/4210216639 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7ff0f40b5680 0x7ff0f40d5b30 unknown :-1 s=CLOSED pgs=308 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:53.461 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.457+0000 7ff1255f86c0 1 --2- 10.20.193.27:0/4210216639 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff120122ae0 0x7ff120122f80 unknown :-1 s=CLOSED pgs=488 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:53.461 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.457+0000 7ff1255f86c0 1 --2- 10.20.193.27:0/4210216639 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff120115440 0x7ff12011d720 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:53.461 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.457+0000 7ff1255f86c0 1 --2- 10.20.193.27:0/4210216639 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff12010f230 0x7ff12011d1e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:53.461 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.457+0000 7ff1255f86c0 1 -- 10.20.193.27:0/4210216639 >> 10.20.193.27:0/4210216639 conn(0x7ff1200ecb30 msgr2=0x7ff12010cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:53.461 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.457+0000 7ff1255f86c0 1 -- 10.20.193.27:0/4210216639 shutdown_connections 2026-02-13T02:26:53.461 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:53.457+0000 7ff1255f86c0 1 -- 10.20.193.27:0/4210216639 wait complete. 2026-02-13T02:26:54.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:54 trial028 ceph-mon[26701]: pgmap v329: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:26:54.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:54 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/4210216639' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:54.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:54 trial107 ceph-mon[27422]: pgmap v329: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:26:54.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:54 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/4210216639' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:54.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:54 trial027 ceph-mon[24567]: pgmap v329: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:26:54.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:54 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/4210216639' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:56.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:56 trial028 ceph-mon[26701]: pgmap v330: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:56.500 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:26:56.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:56 trial107 ceph-mon[27422]: pgmap v330: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:56.525 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:56 trial027 ceph-mon[24567]: pgmap v330: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:56.620 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:26:56.740 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.735+0000 7f9b891ad6c0 1 -- 10.20.193.27:0/4249038669 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9b8410fbf0 msgr2=0x7f9b84114c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:56.740 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.735+0000 7f9b891ad6c0 1 --2- 10.20.193.27:0/4249038669 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9b8410fbf0 0x7f9b84114c30 secure :-1 s=READY pgs=489 gs=1 cs=0 l=1 c_cookie=13500034322195963272 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9b6c0099a0 tx=0x7f9b6c02ed20 comp rx=0 tx=0).stop 2026-02-13T02:26:56.740 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.736+0000 7f9b891ad6c0 1 -- 10.20.193.27:0/4249038669 shutdown_connections 2026-02-13T02:26:56.740 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.736+0000 7f9b891ad6c0 1 --2- 10.20.193.27:0/4249038669 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9b84115410 0x7f9b84175db0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:56.740 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.736+0000 7f9b891ad6c0 1 --2- 10.20.193.27:0/4249038669 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9b8410fbf0 0x7f9b84114c30 unknown :-1 s=CLOSED pgs=489 gs=1 cs=0 l=1 c_cookie=13500034322195963272 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:56.741 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.736+0000 7f9b891ad6c0 1 --2- 10.20.193.27:0/4249038669 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9b8410f200 0x7f9b8410f5d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:56.741 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.736+0000 7f9b891ad6c0 1 -- 10.20.193.27:0/4249038669 >> 10.20.193.27:0/4249038669 conn(0x7f9b840ecb00 msgr2=0x7f9b8410cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:56.741 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.736+0000 7f9b891ad6c0 1 -- 10.20.193.27:0/4249038669 shutdown_connections 2026-02-13T02:26:56.741 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.736+0000 7f9b891ad6c0 1 -- 10.20.193.27:0/4249038669 wait complete. 2026-02-13T02:26:56.741 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.736+0000 7f9b891ad6c0 1 Processor -- start 2026-02-13T02:26:56.741 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.736+0000 7f9b891ad6c0 1 -- start start 2026-02-13T02:26:56.741 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.736+0000 7f9b891ad6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9b8410f200 0x7f9b840f4c40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:56.741 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.736+0000 7f9b891ad6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9b84115410 0x7f9b84175cb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:56.741 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.736+0000 7f9b891ad6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9b840f5270 0x7f9b840fa540 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:56.741 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.736+0000 7f9b891ad6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9b840f5960 con 0x7f9b84115410 2026-02-13T02:26:56.741 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.736+0000 7f9b891ad6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9b840faa80 con 0x7f9b840f5270 2026-02-13T02:26:56.742 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.736+0000 7f9b891ad6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9b840fac40 con 0x7f9b8410f200 2026-02-13T02:26:56.742 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.736+0000 7f9b825756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9b84115410 0x7f9b84175cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:56.742 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.736+0000 7f9b825756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9b84115410 0x7f9b84175cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:48620/0 (socket says 10.20.193.27:48620) 2026-02-13T02:26:56.742 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.736+0000 7f9b825756c0 1 -- 10.20.193.27:0/2667355461 learned_addr learned my addr 10.20.193.27:0/2667355461 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:26:56.742 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.736+0000 7f9b82d766c0 1 --2- 10.20.193.27:0/2667355461 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9b8410f200 0x7f9b840f4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:56.742 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.736+0000 7f9b835776c0 1 --2- 10.20.193.27:0/2667355461 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9b840f5270 0x7f9b840fa540 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:56.742 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.736+0000 7f9b825756c0 1 -- 10.20.193.27:0/2667355461 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9b8410f200 msgr2=0x7f9b840f4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:56.742 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.736+0000 7f9b825756c0 1 --2- 10.20.193.27:0/2667355461 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9b8410f200 0x7f9b840f4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:56.742 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.736+0000 7f9b825756c0 1 -- 10.20.193.27:0/2667355461 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9b840f5270 msgr2=0x7f9b840fa540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:56.742 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.736+0000 7f9b825756c0 1 --2- 10.20.193.27:0/2667355461 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9b840f5270 0x7f9b840fa540 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:56.742 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.736+0000 7f9b825756c0 1 -- 10.20.193.27:0/2667355461 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b6c009590 con 0x7f9b84115410 2026-02-13T02:26:56.742 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.736+0000 7f9b825756c0 1 --2- 10.20.193.27:0/2667355461 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9b84115410 0x7f9b84175cb0 secure :-1 s=READY pgs=490 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9b6c004830 tx=0x7f9b6c02fbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:56.743 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.736+0000 7f9b63fff6c0 1 -- 10.20.193.27:0/2667355461 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9b6c002ae0 con 0x7f9b84115410 2026-02-13T02:26:56.743 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.736+0000 7f9b63fff6c0 1 -- 10.20.193.27:0/2667355461 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9b6c002c40 con 0x7f9b84115410 2026-02-13T02:26:56.743 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.736+0000 7f9b891ad6c0 1 -- 10.20.193.27:0/2667355461 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b840faec0 con 0x7f9b84115410 2026-02-13T02:26:56.743 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.737+0000 7f9b63fff6c0 1 -- 10.20.193.27:0/2667355461 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9b6c0362f0 con 0x7f9b84115410 2026-02-13T02:26:56.743 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.737+0000 7f9b891ad6c0 1 -- 10.20.193.27:0/2667355461 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b840fb360 con 0x7f9b84115410 2026-02-13T02:26:56.743 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.737+0000 7f9b891ad6c0 1 -- 10.20.193.27:0/2667355461 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b841104d0 con 0x7f9b84115410 2026-02-13T02:26:56.744 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.738+0000 7f9b63fff6c0 1 -- 10.20.193.27:0/2667355461 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f9b6c007810 con 0x7f9b84115410 2026-02-13T02:26:56.744 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.739+0000 7f9b63fff6c0 1 --2- 10.20.193.27:0/2667355461 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9b580b5680 0x7f9b580d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:26:56.744 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.739+0000 7f9b63fff6c0 1 -- 10.20.193.27:0/2667355461 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f9b6c107ca0 con 0x7f9b84115410 2026-02-13T02:26:56.744 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.739+0000 7f9b82d766c0 1 --2- 10.20.193.27:0/2667355461 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9b580b5680 0x7f9b580d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:26:56.745 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.739+0000 7f9b63fff6c0 1 -- 10.20.193.27:0/2667355461 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f9b6c0ca720 con 0x7f9b84115410 2026-02-13T02:26:56.745 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.739+0000 7f9b82d766c0 1 --2- 10.20.193.27:0/2667355461 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9b580b5680 0x7f9b580d5b30 secure :-1 s=READY pgs=309 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9b78005980 tx=0x7f9b78005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:26:56.858 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.855+0000 7f9b891ad6c0 1 -- 10.20.193.27:0/2667355461 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f9b8410fbf0 con 0x7f9b84115410 2026-02-13T02:26:56.858 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.856+0000 7f9b63fff6c0 1 -- 10.20.193.27:0/2667355461 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f9b6c0cd8d0 con 0x7f9b84115410 2026-02-13T02:26:56.858 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:26:56.859 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:26:56.860 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.857+0000 7f9b891ad6c0 1 -- 10.20.193.27:0/2667355461 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9b580b5680 msgr2=0x7f9b580d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:56.860 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.857+0000 7f9b891ad6c0 1 --2- 10.20.193.27:0/2667355461 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9b580b5680 0x7f9b580d5b30 secure :-1 s=READY pgs=309 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9b78005980 tx=0x7f9b78005910 comp rx=0 tx=0).stop 2026-02-13T02:26:56.860 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.857+0000 7f9b891ad6c0 1 -- 10.20.193.27:0/2667355461 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9b84115410 msgr2=0x7f9b84175cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:26:56.860 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.857+0000 7f9b891ad6c0 1 --2- 10.20.193.27:0/2667355461 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9b84115410 0x7f9b84175cb0 secure :-1 s=READY pgs=490 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9b6c004830 tx=0x7f9b6c02fbf0 comp rx=0 tx=0).stop 2026-02-13T02:26:56.860 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.857+0000 7f9b891ad6c0 1 -- 10.20.193.27:0/2667355461 shutdown_connections 2026-02-13T02:26:56.860 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.857+0000 7f9b891ad6c0 1 --2- 10.20.193.27:0/2667355461 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f9b580b5680 0x7f9b580d5b30 unknown :-1 s=CLOSED pgs=309 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:56.860 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.857+0000 7f9b891ad6c0 1 --2- 10.20.193.27:0/2667355461 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9b840f5270 0x7f9b840fa540 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:56.860 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.857+0000 7f9b891ad6c0 1 --2- 10.20.193.27:0/2667355461 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f9b84115410 0x7f9b84175cb0 unknown :-1 s=CLOSED pgs=490 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:56.860 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.857+0000 7f9b891ad6c0 1 --2- 10.20.193.27:0/2667355461 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f9b8410f200 0x7f9b840f4c40 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:26:56.860 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.857+0000 7f9b891ad6c0 1 -- 10.20.193.27:0/2667355461 >> 10.20.193.27:0/2667355461 conn(0x7f9b840ecb00 msgr2=0x7f9b84111bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:26:56.861 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.857+0000 7f9b891ad6c0 1 -- 10.20.193.27:0/2667355461 shutdown_connections 2026-02-13T02:26:56.861 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:26:56.857+0000 7f9b891ad6c0 1 -- 10.20.193.27:0/2667355461 wait complete. 2026-02-13T02:26:57.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:57 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2667355461' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:57.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:57 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2667355461' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:57.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:57 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2667355461' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:26:58.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:26:58 trial028 ceph-mon[26701]: pgmap v331: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:58.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:26:58 trial107 ceph-mon[27422]: pgmap v331: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:58.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:26:58 trial027 ceph-mon[24567]: pgmap v331: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:26:59.900 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:27:00.015 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:27:00.135 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.130+0000 7ff006af06c0 1 -- 10.20.193.27:0/1176963013 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff00010f200 msgr2=0x7ff00010f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:00.135 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.130+0000 7ff006af06c0 1 --2- 10.20.193.27:0/1176963013 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff00010f200 0x7ff00010f5d0 secure :-1 s=READY pgs=491 gs=1 cs=0 l=1 c_cookie=11171480062233813433 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7feff00099a0 tx=0x7feff002ed20 comp rx=0 tx=0).stop 2026-02-13T02:27:00.135 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.130+0000 7ff006af06c0 1 -- 10.20.193.27:0/1176963013 shutdown_connections 2026-02-13T02:27:00.135 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.130+0000 7ff006af06c0 1 --2- 10.20.193.27:0/1176963013 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff000115410 0x7ff000175db0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:00.135 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.130+0000 7ff006af06c0 1 --2- 10.20.193.27:0/1176963013 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff00010fbf0 0x7ff000114c30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:00.135 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.130+0000 7ff006af06c0 1 --2- 10.20.193.27:0/1176963013 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff00010f200 0x7ff00010f5d0 unknown :-1 s=CLOSED pgs=491 gs=1 cs=0 l=1 c_cookie=11171480062233813433 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:00.135 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.130+0000 7ff006af06c0 1 -- 10.20.193.27:0/1176963013 >> 10.20.193.27:0/1176963013 conn(0x7ff0000ecb00 msgr2=0x7ff00010cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:27:00.136 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.130+0000 7ff006af06c0 1 -- 10.20.193.27:0/1176963013 shutdown_connections 2026-02-13T02:27:00.136 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.130+0000 7ff006af06c0 1 -- 10.20.193.27:0/1176963013 wait complete. 2026-02-13T02:27:00.136 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.130+0000 7ff006af06c0 1 Processor -- start 2026-02-13T02:27:00.136 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.131+0000 7ff006af06c0 1 -- start start 2026-02-13T02:27:00.136 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.131+0000 7ff006af06c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff00010f200 0x7ff00011d2e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:00.136 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.131+0000 7ff006af06c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff000115410 0x7ff00011d820 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:00.136 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.131+0000 7ff006af06c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff000122be0 0x7ff000123080 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:00.136 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.131+0000 7ff006af06c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff0001235f0 con 0x7ff00010f200 2026-02-13T02:27:00.136 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.131+0000 7ff006af06c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff000123760 con 0x7ff000115410 2026-02-13T02:27:00.136 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.131+0000 7ff006af06c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff000123920 con 0x7ff000122be0 2026-02-13T02:27:00.136 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.131+0000 7ff0048656c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff00010f200 0x7ff00011d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:00.137 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.131+0000 7ff0048656c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff00010f200 0x7ff00011d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:53262/0 (socket says 10.20.193.27:53262) 2026-02-13T02:27:00.137 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.131+0000 7ff0048656c0 1 -- 10.20.193.27:0/3811319811 learned_addr learned my addr 10.20.193.27:0/3811319811 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:27:00.137 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.131+0000 7ff0050666c0 1 --2- 10.20.193.27:0/3811319811 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff000122be0 0x7ff000123080 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:00.137 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.131+0000 7feff7fff6c0 1 --2- 10.20.193.27:0/3811319811 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff000115410 0x7ff00011d820 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:00.137 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.131+0000 7ff0048656c0 1 -- 10.20.193.27:0/3811319811 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff000122be0 msgr2=0x7ff000123080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:00.137 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.131+0000 7ff0048656c0 1 --2- 10.20.193.27:0/3811319811 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff000122be0 0x7ff000123080 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:00.137 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.131+0000 7ff0048656c0 1 -- 10.20.193.27:0/3811319811 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff000115410 msgr2=0x7ff00011d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:00.137 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.131+0000 7ff0048656c0 1 --2- 10.20.193.27:0/3811319811 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff000115410 0x7ff00011d820 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:00.137 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.131+0000 7ff0048656c0 1 -- 10.20.193.27:0/3811319811 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff0009590 con 0x7ff00010f200 2026-02-13T02:27:00.137 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.131+0000 7ff0050666c0 1 --2- 10.20.193.27:0/3811319811 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff000122be0 0x7ff000123080 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:27:00.137 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.131+0000 7ff0048656c0 1 --2- 10.20.193.27:0/3811319811 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff00010f200 0x7ff00011d2e0 secure :-1 s=READY pgs=492 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7feff002f230 tx=0x7feff002fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:27:00.137 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.131+0000 7feff5ffb6c0 1 -- 10.20.193.27:0/3811319811 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7feff0002ba0 con 0x7ff00010f200 2026-02-13T02:27:00.138 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.131+0000 7ff006af06c0 1 -- 10.20.193.27:0/3811319811 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff000163d70 con 0x7ff00010f200 2026-02-13T02:27:00.138 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.131+0000 7feff5ffb6c0 1 -- 10.20.193.27:0/3811319811 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7feff0002d00 con 0x7ff00010f200 2026-02-13T02:27:00.138 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.131+0000 7feff5ffb6c0 1 -- 10.20.193.27:0/3811319811 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7feff00372f0 con 0x7ff00010f200 2026-02-13T02:27:00.138 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.131+0000 7ff006af06c0 1 -- 10.20.193.27:0/3811319811 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0001641a0 con 0x7ff00010f200 2026-02-13T02:27:00.138 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.132+0000 7ff006af06c0 1 -- 10.20.193.27:0/3811319811 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0001104d0 con 0x7ff00010f200 2026-02-13T02:27:00.139 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.133+0000 7feff5ffb6c0 1 -- 10.20.193.27:0/3811319811 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7feff00078d0 con 0x7ff00010f200 2026-02-13T02:27:00.139 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.134+0000 7feff5ffb6c0 1 --2- 10.20.193.27:0/3811319811 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fefdc0b5600 0x7fefdc0d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:00.139 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.134+0000 7feff7fff6c0 1 --2- 10.20.193.27:0/3811319811 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fefdc0b5600 0x7fefdc0d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:00.139 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.134+0000 7feff5ffb6c0 1 -- 10.20.193.27:0/3811319811 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7feff0107c20 con 0x7ff00010f200 2026-02-13T02:27:00.139 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.134+0000 7feff7fff6c0 1 --2- 10.20.193.27:0/3811319811 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fefdc0b5600 0x7fefdc0d5ab0 secure :-1 s=READY pgs=310 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fefe8005fd0 tx=0x7fefe8005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:27:00.139 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.134+0000 7feff5ffb6c0 1 -- 10.20.193.27:0/3811319811 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7feff00ce100 con 0x7ff00010f200 2026-02-13T02:27:00.250 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.247+0000 7ff006af06c0 1 -- 10.20.193.27:0/3811319811 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff00010f5d0 con 0x7ff00010f200 2026-02-13T02:27:00.250 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.247+0000 7feff5ffb6c0 1 -- 10.20.193.27:0/3811319811 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7feff0094760 con 0x7ff00010f200 2026-02-13T02:27:00.250 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:27:00.250 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:27:00.251 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.248+0000 7ff006af06c0 1 -- 10.20.193.27:0/3811319811 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fefdc0b5600 msgr2=0x7fefdc0d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:00.251 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.248+0000 7ff006af06c0 1 --2- 10.20.193.27:0/3811319811 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fefdc0b5600 0x7fefdc0d5ab0 secure :-1 s=READY pgs=310 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fefe8005fd0 tx=0x7fefe8005910 comp rx=0 tx=0).stop 2026-02-13T02:27:00.252 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.248+0000 7ff006af06c0 1 -- 10.20.193.27:0/3811319811 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff00010f200 msgr2=0x7ff00011d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:00.252 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.248+0000 7ff006af06c0 1 --2- 10.20.193.27:0/3811319811 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff00010f200 0x7ff00011d2e0 secure :-1 s=READY pgs=492 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7feff002f230 tx=0x7feff002fd40 comp rx=0 tx=0).stop 2026-02-13T02:27:00.252 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.248+0000 7ff006af06c0 1 -- 10.20.193.27:0/3811319811 shutdown_connections 2026-02-13T02:27:00.252 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.248+0000 7ff006af06c0 1 --2- 10.20.193.27:0/3811319811 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fefdc0b5600 0x7fefdc0d5ab0 unknown :-1 s=CLOSED pgs=310 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:00.252 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.248+0000 7ff006af06c0 1 --2- 10.20.193.27:0/3811319811 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7ff000122be0 0x7ff000123080 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:00.252 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.248+0000 7ff006af06c0 1 --2- 10.20.193.27:0/3811319811 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff000115410 0x7ff00011d820 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:00.252 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.248+0000 7ff006af06c0 1 --2- 10.20.193.27:0/3811319811 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7ff00010f200 0x7ff00011d2e0 unknown :-1 s=CLOSED pgs=492 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:00.252 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.248+0000 7ff006af06c0 1 -- 10.20.193.27:0/3811319811 >> 10.20.193.27:0/3811319811 conn(0x7ff0000ecb00 msgr2=0x7ff00010cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:27:00.252 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.248+0000 7ff006af06c0 1 -- 10.20.193.27:0/3811319811 shutdown_connections 2026-02-13T02:27:00.252 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:00.248+0000 7ff006af06c0 1 -- 10.20.193.27:0/3811319811 wait complete. 2026-02-13T02:27:00.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:00 trial028 ceph-mon[26701]: pgmap v332: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:27:00.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:00 trial107 ceph-mon[27422]: pgmap v332: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:27:00.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:00 trial027 ceph-mon[24567]: pgmap v332: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:27:01.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:01 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3811319811' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:01.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:01 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3811319811' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:01.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:01 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3811319811' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:02.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:02 trial028 ceph-mon[26701]: pgmap v333: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:27:02.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:02 trial107 ceph-mon[27422]: pgmap v333: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:27:02.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:02 trial027 ceph-mon[24567]: pgmap v333: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:27:03.293 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:27:03.412 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:27:03.527 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.522+0000 7f020f3b96c0 1 -- 10.20.193.27:0/3842731426 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0208115410 msgr2=0x7f0208175db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:03.527 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.522+0000 7f020f3b96c0 1 --2- 10.20.193.27:0/3842731426 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0208115410 0x7f0208175db0 secure :-1 s=READY pgs=493 gs=1 cs=0 l=1 c_cookie=11202028578304688655 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f020400aa70 tx=0x7f020402fde0 comp rx=0 tx=0).stop 2026-02-13T02:27:03.527 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.522+0000 7f020f3b96c0 1 -- 10.20.193.27:0/3842731426 shutdown_connections 2026-02-13T02:27:03.527 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.522+0000 7f020f3b96c0 1 --2- 10.20.193.27:0/3842731426 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f0208115410 0x7f0208175db0 unknown :-1 s=CLOSED pgs=493 gs=1 cs=0 l=1 c_cookie=11202028578304688655 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:03.527 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.522+0000 7f020f3b96c0 1 --2- 10.20.193.27:0/3842731426 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f020810fbf0 0x7f0208114c30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:03.527 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.522+0000 7f020f3b96c0 1 --2- 10.20.193.27:0/3842731426 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f020810f200 0x7f020810f5d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:03.528 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.522+0000 7f020f3b96c0 1 -- 10.20.193.27:0/3842731426 >> 10.20.193.27:0/3842731426 conn(0x7f02080ecb00 msgr2=0x7f020810cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:27:03.528 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.522+0000 7f020f3b96c0 1 -- 10.20.193.27:0/3842731426 shutdown_connections 2026-02-13T02:27:03.528 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.522+0000 7f020f3b96c0 1 -- 10.20.193.27:0/3842731426 wait complete. 2026-02-13T02:27:03.528 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.523+0000 7f020f3b96c0 1 Processor -- start 2026-02-13T02:27:03.528 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.523+0000 7f020f3b96c0 1 -- start start 2026-02-13T02:27:03.528 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.523+0000 7f020f3b96c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f020810f200 0x7f020811d290 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:03.528 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.523+0000 7f020f3b96c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0208115410 0x7f020811d7d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:03.528 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.523+0000 7f020f3b96c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f0208122b90 0x7f0208123030 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:03.528 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.523+0000 7f020f3b96c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f02081235a0 con 0x7f020810f200 2026-02-13T02:27:03.528 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.523+0000 7f020f3b96c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0208123710 con 0x7f0208115410 2026-02-13T02:27:03.528 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.523+0000 7f020f3b96c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f02081238d0 con 0x7f0208122b90 2026-02-13T02:27:03.528 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.523+0000 7f020d12a6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f020810f200 0x7f020811d290 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:03.529 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.523+0000 7f020d12a6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f020810f200 0x7f020811d290 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:53284/0 (socket says 10.20.193.27:53284) 2026-02-13T02:27:03.529 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.523+0000 7f020d12a6c0 1 -- 10.20.193.27:0/492392658 learned_addr learned my addr 10.20.193.27:0/492392658 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:27:03.529 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.523+0000 7f020c9296c0 1 --2- 10.20.193.27:0/492392658 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0208115410 0x7f020811d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:03.529 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.523+0000 7f020d92b6c0 1 --2- 10.20.193.27:0/492392658 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f0208122b90 0x7f0208123030 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:03.529 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.523+0000 7f020d12a6c0 1 -- 10.20.193.27:0/492392658 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f0208122b90 msgr2=0x7f0208123030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:03.529 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.523+0000 7f020d12a6c0 1 --2- 10.20.193.27:0/492392658 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f0208122b90 0x7f0208123030 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:03.529 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.523+0000 7f020d12a6c0 1 -- 10.20.193.27:0/492392658 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0208115410 msgr2=0x7f020811d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:03.529 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.523+0000 7f020d12a6c0 1 --2- 10.20.193.27:0/492392658 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0208115410 0x7f020811d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:03.529 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.523+0000 7f020d12a6c0 1 -- 10.20.193.27:0/492392658 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f020400a660 con 0x7f020810f200 2026-02-13T02:27:03.529 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.523+0000 7f020d92b6c0 1 --2- 10.20.193.27:0/492392658 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f0208122b90 0x7f0208123030 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:27:03.529 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.523+0000 7f020d12a6c0 1 --2- 10.20.193.27:0/492392658 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f020810f200 0x7f020811d290 secure :-1 s=READY pgs=494 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f01f8005dc0 tx=0x7f01f800bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:27:03.530 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.523+0000 7f01f67fc6c0 1 -- 10.20.193.27:0/492392658 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f01f8004250 con 0x7f020810f200 2026-02-13T02:27:03.530 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.523+0000 7f01f67fc6c0 1 -- 10.20.193.27:0/492392658 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f01f8009070 con 0x7f020810f200 2026-02-13T02:27:03.530 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.523+0000 7f01f67fc6c0 1 -- 10.20.193.27:0/492392658 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f01f800cd70 con 0x7f020810f200 2026-02-13T02:27:03.530 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.523+0000 7f020f3b96c0 1 -- 10.20.193.27:0/492392658 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0208163d80 con 0x7f020810f200 2026-02-13T02:27:03.530 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.523+0000 7f020f3b96c0 1 -- 10.20.193.27:0/492392658 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0208164210 con 0x7f020810f200 2026-02-13T02:27:03.530 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.524+0000 7f020f3b96c0 1 -- 10.20.193.27:0/492392658 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02081104d0 con 0x7f020810f200 2026-02-13T02:27:03.531 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.525+0000 7f01f67fc6c0 1 -- 10.20.193.27:0/492392658 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f01f80028c0 con 0x7f020810f200 2026-02-13T02:27:03.531 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.526+0000 7f01f67fc6c0 1 --2- 10.20.193.27:0/492392658 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f01e40b5680 0x7f01e40d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:03.531 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.526+0000 7f020c9296c0 1 --2- 10.20.193.27:0/492392658 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f01e40b5680 0x7f01e40d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:03.531 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.526+0000 7f01f67fc6c0 1 -- 10.20.193.27:0/492392658 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f01f80e3a90 con 0x7f020810f200 2026-02-13T02:27:03.532 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.526+0000 7f020c9296c0 1 --2- 10.20.193.27:0/492392658 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f01e40b5680 0x7f01e40d5b30 secure :-1 s=READY pgs=311 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0208123c20 tx=0x7f01fc009260 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:27:03.532 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.527+0000 7f01f67fc6c0 1 -- 10.20.193.27:0/492392658 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f01f80a6510 con 0x7f020810f200 2026-02-13T02:27:03.643 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.640+0000 7f020f3b96c0 1 -- 10.20.193.27:0/492392658 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f020810f5d0 con 0x7f020810f200 2026-02-13T02:27:03.643 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.641+0000 7f01f67fc6c0 1 -- 10.20.193.27:0/492392658 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f01f80a96c0 con 0x7f020810f200 2026-02-13T02:27:03.643 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:27:03.644 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:27:03.644 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.642+0000 7f020f3b96c0 1 -- 10.20.193.27:0/492392658 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f01e40b5680 msgr2=0x7f01e40d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:03.645 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.642+0000 7f020f3b96c0 1 --2- 10.20.193.27:0/492392658 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f01e40b5680 0x7f01e40d5b30 secure :-1 s=READY pgs=311 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0208123c20 tx=0x7f01fc009260 comp rx=0 tx=0).stop 2026-02-13T02:27:03.645 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.642+0000 7f020f3b96c0 1 -- 10.20.193.27:0/492392658 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f020810f200 msgr2=0x7f020811d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:03.645 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.642+0000 7f020f3b96c0 1 --2- 10.20.193.27:0/492392658 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f020810f200 0x7f020811d290 secure :-1 s=READY pgs=494 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f01f8005dc0 tx=0x7f01f800bbb0 comp rx=0 tx=0).stop 2026-02-13T02:27:03.645 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.642+0000 7f020f3b96c0 1 -- 10.20.193.27:0/492392658 shutdown_connections 2026-02-13T02:27:03.645 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.642+0000 7f020f3b96c0 1 --2- 10.20.193.27:0/492392658 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f01e40b5680 0x7f01e40d5b30 unknown :-1 s=CLOSED pgs=311 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:03.645 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.642+0000 7f020f3b96c0 1 --2- 10.20.193.27:0/492392658 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f0208122b90 0x7f0208123030 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:03.645 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.642+0000 7f020f3b96c0 1 --2- 10.20.193.27:0/492392658 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0208115410 0x7f020811d7d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:03.645 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.642+0000 7f020f3b96c0 1 --2- 10.20.193.27:0/492392658 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f020810f200 0x7f020811d290 unknown :-1 s=CLOSED pgs=494 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:03.645 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.642+0000 7f020f3b96c0 1 -- 10.20.193.27:0/492392658 >> 10.20.193.27:0/492392658 conn(0x7f02080ecb00 msgr2=0x7f020810cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:27:03.645 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.642+0000 7f020f3b96c0 1 -- 10.20.193.27:0/492392658 shutdown_connections 2026-02-13T02:27:03.645 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:03.642+0000 7f020f3b96c0 1 -- 10.20.193.27:0/492392658 wait complete. 2026-02-13T02:27:04.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:04 trial028 ceph-mon[26701]: pgmap v334: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:27:04.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:04 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/492392658' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:04.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:04 trial107 ceph-mon[27422]: pgmap v334: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:27:04.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:04 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/492392658' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:04.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:04 trial027 ceph-mon[24567]: pgmap v334: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:27:04.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:04 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/492392658' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:06.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:06 trial028 ceph-mon[26701]: pgmap v335: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:27:06.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:06 trial107 ceph-mon[27422]: pgmap v335: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:27:06.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:06 trial027 ceph-mon[24567]: pgmap v335: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:27:06.687 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:27:06.804 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:27:06.926 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.922+0000 7f4a1c8406c0 1 -- 10.20.193.27:0/3546214898 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4a1410f230 msgr2=0x7f4a1410f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:06.926 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.922+0000 7f4a1c8406c0 1 --2- 10.20.193.27:0/3546214898 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4a1410f230 0x7f4a1410f600 secure :-1 s=READY pgs=495 gs=1 cs=0 l=1 c_cookie=8436639667553593375 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4a0c0099a0 tx=0x7f4a0c02ed20 comp rx=0 tx=0).stop 2026-02-13T02:27:06.926 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.922+0000 7f4a1c8406c0 1 -- 10.20.193.27:0/3546214898 shutdown_connections 2026-02-13T02:27:06.926 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.922+0000 7f4a1c8406c0 1 --2- 10.20.193.27:0/3546214898 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4a14115440 0x7f4a14175de0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:06.927 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.922+0000 7f4a1c8406c0 1 --2- 10.20.193.27:0/3546214898 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f4a1410fc20 0x7f4a14114c60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:06.927 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.922+0000 7f4a1c8406c0 1 --2- 10.20.193.27:0/3546214898 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4a1410f230 0x7f4a1410f600 unknown :-1 s=CLOSED pgs=495 gs=1 cs=0 l=1 c_cookie=8436639667553593375 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:06.927 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.922+0000 7f4a1c8406c0 1 -- 10.20.193.27:0/3546214898 >> 10.20.193.27:0/3546214898 conn(0x7f4a140ecb30 msgr2=0x7f4a1410cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:27:06.927 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.922+0000 7f4a1c8406c0 1 -- 10.20.193.27:0/3546214898 shutdown_connections 2026-02-13T02:27:06.927 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.922+0000 7f4a1c8406c0 1 -- 10.20.193.27:0/3546214898 wait complete. 2026-02-13T02:27:06.927 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.922+0000 7f4a1c8406c0 1 Processor -- start 2026-02-13T02:27:06.927 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.922+0000 7f4a1c8406c0 1 -- start start 2026-02-13T02:27:06.927 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.922+0000 7f4a1c8406c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4a1410f230 0x7f4a1411d300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:06.927 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.922+0000 7f4a1c8406c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f4a14115440 0x7f4a1411d840 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:06.927 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.922+0000 7f4a1c8406c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4a14122c00 0x7f4a141230a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:06.927 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.922+0000 7f4a1c8406c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4a14123610 con 0x7f4a1410f230 2026-02-13T02:27:06.927 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.922+0000 7f4a1c8406c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4a14123780 con 0x7f4a14122c00 2026-02-13T02:27:06.928 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.922+0000 7f4a1c8406c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4a14123940 con 0x7f4a14115440 2026-02-13T02:27:06.928 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.923+0000 7f4a1a5b56c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4a1410f230 0x7f4a1411d300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:06.928 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.923+0000 7f4a1a5b56c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4a1410f230 0x7f4a1411d300 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:53310/0 (socket says 10.20.193.27:53310) 2026-02-13T02:27:06.928 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.923+0000 7f4a1a5b56c0 1 -- 10.20.193.27:0/1044604213 learned_addr learned my addr 10.20.193.27:0/1044604213 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:27:06.928 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.923+0000 7f4a19db46c0 1 --2- 10.20.193.27:0/1044604213 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f4a14115440 0x7f4a1411d840 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:06.928 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.923+0000 7f4a1adb66c0 1 --2- 10.20.193.27:0/1044604213 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4a14122c00 0x7f4a141230a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:06.928 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.923+0000 7f4a1a5b56c0 1 -- 10.20.193.27:0/1044604213 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f4a14115440 msgr2=0x7f4a1411d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:06.928 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.923+0000 7f4a1a5b56c0 1 --2- 10.20.193.27:0/1044604213 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f4a14115440 0x7f4a1411d840 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:06.928 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.923+0000 7f4a1a5b56c0 1 -- 10.20.193.27:0/1044604213 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4a14122c00 msgr2=0x7f4a141230a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:06.928 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.923+0000 7f4a1a5b56c0 1 --2- 10.20.193.27:0/1044604213 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4a14122c00 0x7f4a141230a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:06.928 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.923+0000 7f4a1a5b56c0 1 -- 10.20.193.27:0/1044604213 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0c009590 con 0x7f4a1410f230 2026-02-13T02:27:06.928 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.923+0000 7f4a1adb66c0 1 --2- 10.20.193.27:0/1044604213 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4a14122c00 0x7f4a141230a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:27:06.929 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.923+0000 7f4a1a5b56c0 1 --2- 10.20.193.27:0/1044604213 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4a1410f230 0x7f4a1411d300 secure :-1 s=READY pgs=496 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4a0c02f230 tx=0x7f4a0c02fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:27:06.929 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.923+0000 7f49ff7fe6c0 1 -- 10.20.193.27:0/1044604213 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4a0c002ba0 con 0x7f4a1410f230 2026-02-13T02:27:06.929 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.923+0000 7f49ff7fe6c0 1 -- 10.20.193.27:0/1044604213 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4a0c002d00 con 0x7f4a1410f230 2026-02-13T02:27:06.929 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.923+0000 7f4a1c8406c0 1 -- 10.20.193.27:0/1044604213 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a14163d90 con 0x7f4a1410f230 2026-02-13T02:27:06.929 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.923+0000 7f49ff7fe6c0 1 -- 10.20.193.27:0/1044604213 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4a0c0372f0 con 0x7f4a1410f230 2026-02-13T02:27:06.929 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.923+0000 7f4a1c8406c0 1 -- 10.20.193.27:0/1044604213 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a141641c0 con 0x7f4a1410f230 2026-02-13T02:27:06.929 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.923+0000 7f4a1c8406c0 1 -- 10.20.193.27:0/1044604213 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a14110500 con 0x7f4a1410f230 2026-02-13T02:27:06.930 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.924+0000 7f49ff7fe6c0 1 -- 10.20.193.27:0/1044604213 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f4a0c0078d0 con 0x7f4a1410f230 2026-02-13T02:27:06.930 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.925+0000 7f49ff7fe6c0 1 --2- 10.20.193.27:0/1044604213 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f49f00b5600 0x7f49f00d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:06.930 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.925+0000 7f49ff7fe6c0 1 -- 10.20.193.27:0/1044604213 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f4a0c107c20 con 0x7f4a1410f230 2026-02-13T02:27:06.930 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.925+0000 7f4a19db46c0 1 --2- 10.20.193.27:0/1044604213 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f49f00b5600 0x7f49f00d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:06.930 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.925+0000 7f4a19db46c0 1 --2- 10.20.193.27:0/1044604213 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f49f00b5600 0x7f49f00d5ab0 secure :-1 s=READY pgs=312 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4a14123c90 tx=0x7f4a00005ee0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:27:06.930 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:06.925+0000 7f49ff7fe6c0 1 -- 10.20.193.27:0/1044604213 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f4a0c0ca6a0 con 0x7f4a1410f230 2026-02-13T02:27:07.044 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:07.042+0000 7f4a1c8406c0 1 -- 10.20.193.27:0/1044604213 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4a1410f600 con 0x7f4a1410f230 2026-02-13T02:27:07.044 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:07.042+0000 7f49ff7fe6c0 1 -- 10.20.193.27:0/1044604213 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f4a0c0cd850 con 0x7f4a1410f230 2026-02-13T02:27:07.044 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:27:07.045 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:27:07.046 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:07.043+0000 7f4a1c8406c0 1 -- 10.20.193.27:0/1044604213 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f49f00b5600 msgr2=0x7f49f00d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:07.046 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:07.043+0000 7f4a1c8406c0 1 --2- 10.20.193.27:0/1044604213 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f49f00b5600 0x7f49f00d5ab0 secure :-1 s=READY pgs=312 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4a14123c90 tx=0x7f4a00005ee0 comp rx=0 tx=0).stop 2026-02-13T02:27:07.046 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:07.043+0000 7f4a1c8406c0 1 -- 10.20.193.27:0/1044604213 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4a1410f230 msgr2=0x7f4a1411d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:07.046 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:07.043+0000 7f4a1c8406c0 1 --2- 10.20.193.27:0/1044604213 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4a1410f230 0x7f4a1411d300 secure :-1 s=READY pgs=496 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4a0c02f230 tx=0x7f4a0c02fd40 comp rx=0 tx=0).stop 2026-02-13T02:27:07.046 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:07.043+0000 7f4a1c8406c0 1 -- 10.20.193.27:0/1044604213 shutdown_connections 2026-02-13T02:27:07.046 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:07.043+0000 7f4a1c8406c0 1 --2- 10.20.193.27:0/1044604213 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f49f00b5600 0x7f49f00d5ab0 unknown :-1 s=CLOSED pgs=312 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:07.046 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:07.043+0000 7f4a1c8406c0 1 --2- 10.20.193.27:0/1044604213 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4a14122c00 0x7f4a141230a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:07.046 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:07.043+0000 7f4a1c8406c0 1 --2- 10.20.193.27:0/1044604213 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f4a14115440 0x7f4a1411d840 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:07.046 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:07.043+0000 7f4a1c8406c0 1 --2- 10.20.193.27:0/1044604213 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f4a1410f230 0x7f4a1411d300 unknown :-1 s=CLOSED pgs=496 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:07.046 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:07.043+0000 7f4a1c8406c0 1 -- 10.20.193.27:0/1044604213 >> 10.20.193.27:0/1044604213 conn(0x7f4a140ecb30 msgr2=0x7f4a1410cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:27:07.047 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:07.043+0000 7f4a1c8406c0 1 -- 10.20.193.27:0/1044604213 shutdown_connections 2026-02-13T02:27:07.047 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:07.043+0000 7f4a1c8406c0 1 -- 10.20.193.27:0/1044604213 wait complete. 2026-02-13T02:27:07.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:07 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1044604213' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:07.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:07 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1044604213' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:07.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:07 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1044604213' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:08.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:08 trial028 ceph-mon[26701]: pgmap v336: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:27:08.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:08 trial107 ceph-mon[27422]: pgmap v336: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:27:08.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:08 trial027 ceph-mon[24567]: pgmap v336: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:27:10.085 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:27:10.201 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:27:10.321 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.318+0000 7f5ebcae06c0 1 -- 10.20.193.27:0/1430059804 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5eb810fbf0 msgr2=0x7f5eb8114c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:10.321 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.318+0000 7f5ebcae06c0 1 --2- 10.20.193.27:0/1430059804 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5eb810fbf0 0x7f5eb8114c30 secure :-1 s=READY pgs=497 gs=1 cs=0 l=1 c_cookie=13589582580663512471 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5ea00099a0 tx=0x7f5ea002ed20 comp rx=0 tx=0).stop 2026-02-13T02:27:10.321 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.318+0000 7f5ebcae06c0 1 -- 10.20.193.27:0/1430059804 shutdown_connections 2026-02-13T02:27:10.322 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.318+0000 7f5ebcae06c0 1 --2- 10.20.193.27:0/1430059804 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5eb8115410 0x7f5eb8175db0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:10.322 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.318+0000 7f5ebcae06c0 1 --2- 10.20.193.27:0/1430059804 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5eb810fbf0 0x7f5eb8114c30 unknown :-1 s=CLOSED pgs=497 gs=1 cs=0 l=1 c_cookie=13589582580663512471 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:10.322 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.318+0000 7f5ebcae06c0 1 --2- 10.20.193.27:0/1430059804 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5eb810f200 0x7f5eb810f5d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:10.322 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.318+0000 7f5ebcae06c0 1 -- 10.20.193.27:0/1430059804 >> 10.20.193.27:0/1430059804 conn(0x7f5eb80ecb00 msgr2=0x7f5eb810cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:27:10.322 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.318+0000 7f5ebcae06c0 1 -- 10.20.193.27:0/1430059804 shutdown_connections 2026-02-13T02:27:10.322 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.318+0000 7f5ebcae06c0 1 -- 10.20.193.27:0/1430059804 wait complete. 2026-02-13T02:27:10.322 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.318+0000 7f5ebcae06c0 1 Processor -- start 2026-02-13T02:27:10.322 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.318+0000 7f5ebcae06c0 1 -- start start 2026-02-13T02:27:10.322 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.318+0000 7f5ebcae06c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5eb810f200 0x7f5eb80f4d00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:10.322 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.318+0000 7f5ebcae06c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5eb8115410 0x7f5eb8175cb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:10.322 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.318+0000 7f5ebcae06c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5eb80f5330 0x7f5eb80fa5b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:10.322 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.318+0000 7f5ebcae06c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb80f5a20 con 0x7f5eb810f200 2026-02-13T02:27:10.323 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.318+0000 7f5ebcae06c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb80faaf0 con 0x7f5eb8115410 2026-02-13T02:27:10.323 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.318+0000 7f5ebcae06c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb80facb0 con 0x7f5eb80f5330 2026-02-13T02:27:10.323 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.318+0000 7f5eb65756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5eb810f200 0x7f5eb80f4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:10.323 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.318+0000 7f5eb65756c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5eb810f200 0x7f5eb80f4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:56150/0 (socket says 10.20.193.27:56150) 2026-02-13T02:27:10.323 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.318+0000 7f5eb65756c0 1 -- 10.20.193.27:0/3548774409 learned_addr learned my addr 10.20.193.27:0/3548774409 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:27:10.323 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.318+0000 7f5eb5d746c0 1 --2- 10.20.193.27:0/3548774409 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5eb8115410 0x7f5eb8175cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:10.323 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.318+0000 7f5eb6d766c0 1 --2- 10.20.193.27:0/3548774409 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5eb80f5330 0x7f5eb80fa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:10.323 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.318+0000 7f5eb65756c0 1 -- 10.20.193.27:0/3548774409 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5eb80f5330 msgr2=0x7f5eb80fa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:10.323 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.318+0000 7f5eb65756c0 1 --2- 10.20.193.27:0/3548774409 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5eb80f5330 0x7f5eb80fa5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:10.323 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.318+0000 7f5eb65756c0 1 -- 10.20.193.27:0/3548774409 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5eb8115410 msgr2=0x7f5eb8175cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:10.323 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.318+0000 7f5eb65756c0 1 --2- 10.20.193.27:0/3548774409 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5eb8115410 0x7f5eb8175cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:10.323 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.319+0000 7f5eb65756c0 1 -- 10.20.193.27:0/3548774409 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea0009590 con 0x7f5eb810f200 2026-02-13T02:27:10.324 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.319+0000 7f5eb6d766c0 1 --2- 10.20.193.27:0/3548774409 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5eb80f5330 0x7f5eb80fa5b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:27:10.324 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.319+0000 7f5eb65756c0 1 --2- 10.20.193.27:0/3548774409 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5eb810f200 0x7f5eb80f4d00 secure :-1 s=READY pgs=498 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5eac00b710 tx=0x7f5eac00bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:27:10.324 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.319+0000 7f5e9f7fe6c0 1 -- 10.20.193.27:0/3548774409 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5eac004250 con 0x7f5eb810f200 2026-02-13T02:27:10.324 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.319+0000 7f5e9f7fe6c0 1 -- 10.20.193.27:0/3548774409 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5eac002d80 con 0x7f5eb810f200 2026-02-13T02:27:10.324 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.319+0000 7f5ebcae06c0 1 -- 10.20.193.27:0/3548774409 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eb80faf90 con 0x7f5eb810f200 2026-02-13T02:27:10.324 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.319+0000 7f5e9f7fe6c0 1 -- 10.20.193.27:0/3548774409 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5eac00cda0 con 0x7f5eb810f200 2026-02-13T02:27:10.324 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.319+0000 7f5ebcae06c0 1 -- 10.20.193.27:0/3548774409 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eb80fb720 con 0x7f5eb810f200 2026-02-13T02:27:10.324 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.319+0000 7f5ebcae06c0 1 -- 10.20.193.27:0/3548774409 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eb81104d0 con 0x7f5eb810f200 2026-02-13T02:27:10.325 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.320+0000 7f5e9f7fe6c0 1 -- 10.20.193.27:0/3548774409 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f5eac0028c0 con 0x7f5eb810f200 2026-02-13T02:27:10.325 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.321+0000 7f5e9f7fe6c0 1 --2- 10.20.193.27:0/3548774409 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f5e900b5600 0x7f5e900d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:10.325 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.321+0000 7f5eb5d746c0 1 --2- 10.20.193.27:0/3548774409 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f5e900b5600 0x7f5e900d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:10.326 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.321+0000 7f5e9f7fe6c0 1 -- 10.20.193.27:0/3548774409 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f5eac0e3b30 con 0x7f5eb810f200 2026-02-13T02:27:10.326 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.321+0000 7f5e9f7fe6c0 1 -- 10.20.193.27:0/3548774409 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f5eac0a9580 con 0x7f5eb810f200 2026-02-13T02:27:10.326 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.322+0000 7f5eb5d746c0 1 --2- 10.20.193.27:0/3548774409 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f5e900b5600 0x7f5e900d5ab0 secure :-1 s=READY pgs=313 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5ea0005b40 tx=0x7f5ea00092a0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:27:10.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:10 trial027 ceph-mon[24567]: pgmap v337: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:27:10.438 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.437+0000 7f5ebcae06c0 1 -- 10.20.193.27:0/3548774409 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5eb80f5e00 con 0x7f5eb810f200 2026-02-13T02:27:10.439 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.437+0000 7f5e9f7fe6c0 1 -- 10.20.193.27:0/3548774409 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f5eac0a9760 con 0x7f5eb810f200 2026-02-13T02:27:10.439 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:27:10.439 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:27:10.440 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.438+0000 7f5ebcae06c0 1 -- 10.20.193.27:0/3548774409 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f5e900b5600 msgr2=0x7f5e900d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:10.440 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.438+0000 7f5ebcae06c0 1 --2- 10.20.193.27:0/3548774409 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f5e900b5600 0x7f5e900d5ab0 secure :-1 s=READY pgs=313 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5ea0005b40 tx=0x7f5ea00092a0 comp rx=0 tx=0).stop 2026-02-13T02:27:10.440 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.438+0000 7f5ebcae06c0 1 -- 10.20.193.27:0/3548774409 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5eb810f200 msgr2=0x7f5eb80f4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:10.440 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.438+0000 7f5ebcae06c0 1 --2- 10.20.193.27:0/3548774409 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5eb810f200 0x7f5eb80f4d00 secure :-1 s=READY pgs=498 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5eac00b710 tx=0x7f5eac00bbe0 comp rx=0 tx=0).stop 2026-02-13T02:27:10.440 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.438+0000 7f5ebcae06c0 1 -- 10.20.193.27:0/3548774409 shutdown_connections 2026-02-13T02:27:10.441 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.438+0000 7f5ebcae06c0 1 --2- 10.20.193.27:0/3548774409 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f5e900b5600 0x7f5e900d5ab0 unknown :-1 s=CLOSED pgs=313 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:10.441 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.438+0000 7f5ebcae06c0 1 --2- 10.20.193.27:0/3548774409 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5eb80f5330 0x7f5eb80fa5b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:10.441 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.438+0000 7f5ebcae06c0 1 --2- 10.20.193.27:0/3548774409 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5eb8115410 0x7f5eb8175cb0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:10.441 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.438+0000 7f5ebcae06c0 1 --2- 10.20.193.27:0/3548774409 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f5eb810f200 0x7f5eb80f4d00 unknown :-1 s=CLOSED pgs=498 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:10.441 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.438+0000 7f5ebcae06c0 1 -- 10.20.193.27:0/3548774409 >> 10.20.193.27:0/3548774409 conn(0x7f5eb80ecb00 msgr2=0x7f5eb810d940 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:27:10.441 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.438+0000 7f5ebcae06c0 1 -- 10.20.193.27:0/3548774409 shutdown_connections 2026-02-13T02:27:10.441 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:10.438+0000 7f5ebcae06c0 1 -- 10.20.193.27:0/3548774409 wait complete. 2026-02-13T02:27:10.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:10 trial028 ceph-mon[26701]: pgmap v337: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:27:10.500 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:10 trial107 ceph-mon[27422]: pgmap v337: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:27:11.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:11 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/3548774409' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:11.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:11 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/3548774409' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:11.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:11 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/3548774409' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:12.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:12 trial028 ceph-mon[26701]: pgmap v338: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:27:12.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:12 trial107 ceph-mon[27422]: pgmap v338: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:27:12.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:12 trial027 ceph-mon[24567]: pgmap v338: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:27:13.480 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:27:13.600 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:27:13.720 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.716+0000 7f6f5b9576c0 1 -- 10.20.193.27:0/2662111232 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f6f54115410 msgr2=0x7f6f54175db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:13.720 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.716+0000 7f6f5b9576c0 1 --2- 10.20.193.27:0/2662111232 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f6f54115410 0x7f6f54175db0 secure :-1 s=READY pgs=499 gs=1 cs=0 l=1 c_cookie=2815424434348776238 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6f5000aa70 tx=0x7f6f5002fde0 comp rx=0 tx=0).stop 2026-02-13T02:27:13.720 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.716+0000 7f6f5b9576c0 1 -- 10.20.193.27:0/2662111232 shutdown_connections 2026-02-13T02:27:13.720 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.716+0000 7f6f5b9576c0 1 --2- 10.20.193.27:0/2662111232 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f6f54115410 0x7f6f54175db0 unknown :-1 s=CLOSED pgs=499 gs=1 cs=0 l=1 c_cookie=2815424434348776238 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:13.720 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.716+0000 7f6f5b9576c0 1 --2- 10.20.193.27:0/2662111232 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6f5410fbf0 0x7f6f54114c30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:13.720 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.716+0000 7f6f5b9576c0 1 --2- 10.20.193.27:0/2662111232 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f6f5410f200 0x7f6f5410f5d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:13.720 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.716+0000 7f6f5b9576c0 1 -- 10.20.193.27:0/2662111232 >> 10.20.193.27:0/2662111232 conn(0x7f6f540ecb00 msgr2=0x7f6f5410cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:27:13.721 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.716+0000 7f6f5b9576c0 1 -- 10.20.193.27:0/2662111232 shutdown_connections 2026-02-13T02:27:13.721 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.716+0000 7f6f5b9576c0 1 -- 10.20.193.27:0/2662111232 wait complete. 2026-02-13T02:27:13.721 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.717+0000 7f6f5b9576c0 1 Processor -- start 2026-02-13T02:27:13.721 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.717+0000 7f6f5b9576c0 1 -- start start 2026-02-13T02:27:13.721 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.717+0000 7f6f5b9576c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f6f5410f200 0x7f6f5411d250 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:13.721 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.717+0000 7f6f5b9576c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f6f54115410 0x7f6f5411d790 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:13.721 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.717+0000 7f6f5b9576c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6f54122b50 0x7f6f54122ff0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:13.721 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.717+0000 7f6f5b9576c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6f54123530 con 0x7f6f54115410 2026-02-13T02:27:13.721 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.717+0000 7f6f5b9576c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6f541236a0 con 0x7f6f54122b50 2026-02-13T02:27:13.721 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.717+0000 7f6f5b9576c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6f54123860 con 0x7f6f5410f200 2026-02-13T02:27:13.721 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.717+0000 7f6f58ec76c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f6f54115410 0x7f6f5411d790 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:13.721 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.717+0000 7f6f58ec76c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f6f54115410 0x7f6f5411d790 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:56164/0 (socket says 10.20.193.27:56164) 2026-02-13T02:27:13.722 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.717+0000 7f6f58ec76c0 1 -- 10.20.193.27:0/1327839890 learned_addr learned my addr 10.20.193.27:0/1327839890 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:27:13.722 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.717+0000 7f6f596c86c0 1 --2- 10.20.193.27:0/1327839890 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f6f5410f200 0x7f6f5411d250 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:13.722 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.717+0000 7f6f59ec96c0 1 --2- 10.20.193.27:0/1327839890 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6f54122b50 0x7f6f54122ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:13.722 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.717+0000 7f6f58ec76c0 1 -- 10.20.193.27:0/1327839890 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f6f5410f200 msgr2=0x7f6f5411d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:13.722 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.717+0000 7f6f58ec76c0 1 --2- 10.20.193.27:0/1327839890 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f6f5410f200 0x7f6f5411d250 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:13.722 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.717+0000 7f6f58ec76c0 1 -- 10.20.193.27:0/1327839890 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6f54122b50 msgr2=0x7f6f54122ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:13.722 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.717+0000 7f6f58ec76c0 1 --2- 10.20.193.27:0/1327839890 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6f54122b50 0x7f6f54122ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:13.722 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.717+0000 7f6f58ec76c0 1 -- 10.20.193.27:0/1327839890 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f5000a660 con 0x7f6f54115410 2026-02-13T02:27:13.722 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.717+0000 7f6f59ec96c0 1 --2- 10.20.193.27:0/1327839890 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6f54122b50 0x7f6f54122ff0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:27:13.722 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.717+0000 7f6f58ec76c0 1 --2- 10.20.193.27:0/1327839890 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f6f54115410 0x7f6f5411d790 secure :-1 s=READY pgs=500 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6f4800b650 tx=0x7f6f4800bb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:27:13.722 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.717+0000 7f6f427fc6c0 1 -- 10.20.193.27:0/1327839890 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6f480044a0 con 0x7f6f54115410 2026-02-13T02:27:13.722 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.717+0000 7f6f5b9576c0 1 -- 10.20.193.27:0/1327839890 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f54163ce0 con 0x7f6f54115410 2026-02-13T02:27:13.723 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.717+0000 7f6f427fc6c0 1 -- 10.20.193.27:0/1327839890 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6f48009030 con 0x7f6f54115410 2026-02-13T02:27:13.723 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.717+0000 7f6f427fc6c0 1 -- 10.20.193.27:0/1327839890 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6f4800cc10 con 0x7f6f54115410 2026-02-13T02:27:13.723 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.717+0000 7f6f5b9576c0 1 -- 10.20.193.27:0/1327839890 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f541641c0 con 0x7f6f54115410 2026-02-13T02:27:13.723 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.718+0000 7f6f5b9576c0 1 -- 10.20.193.27:0/1327839890 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f541104d0 con 0x7f6f54115410 2026-02-13T02:27:13.724 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.719+0000 7f6f427fc6c0 1 -- 10.20.193.27:0/1327839890 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f6f480026b0 con 0x7f6f54115410 2026-02-13T02:27:13.724 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.720+0000 7f6f427fc6c0 1 --2- 10.20.193.27:0/1327839890 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f6f300b5600 0x7f6f300d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:13.724 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.720+0000 7f6f427fc6c0 1 -- 10.20.193.27:0/1327839890 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f6f480e39c0 con 0x7f6f54115410 2026-02-13T02:27:13.724 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.720+0000 7f6f596c86c0 1 --2- 10.20.193.27:0/1327839890 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f6f300b5600 0x7f6f300d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:13.724 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.720+0000 7f6f596c86c0 1 --2- 10.20.193.27:0/1327839890 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f6f300b5600 0x7f6f300d5ab0 secure :-1 s=READY pgs=314 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6f44004750 tx=0x7f6f44005e60 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:27:13.724 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.721+0000 7f6f427fc6c0 1 -- 10.20.193.27:0/1327839890 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f6f480a6480 con 0x7f6f54115410 2026-02-13T02:27:13.839 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.838+0000 7f6f5b9576c0 1 -- 10.20.193.27:0/1327839890 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6f5411e3a0 con 0x7f6f54115410 2026-02-13T02:27:13.840 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.838+0000 7f6f427fc6c0 1 -- 10.20.193.27:0/1327839890 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f6f480a9630 con 0x7f6f54115410 2026-02-13T02:27:13.840 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:27:13.840 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:27:13.841 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.840+0000 7f6f5b9576c0 1 -- 10.20.193.27:0/1327839890 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f6f300b5600 msgr2=0x7f6f300d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:13.841 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.840+0000 7f6f5b9576c0 1 --2- 10.20.193.27:0/1327839890 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f6f300b5600 0x7f6f300d5ab0 secure :-1 s=READY pgs=314 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6f44004750 tx=0x7f6f44005e60 comp rx=0 tx=0).stop 2026-02-13T02:27:13.841 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.840+0000 7f6f5b9576c0 1 -- 10.20.193.27:0/1327839890 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f6f54115410 msgr2=0x7f6f5411d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:13.842 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.840+0000 7f6f5b9576c0 1 --2- 10.20.193.27:0/1327839890 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f6f54115410 0x7f6f5411d790 secure :-1 s=READY pgs=500 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6f4800b650 tx=0x7f6f4800bb20 comp rx=0 tx=0).stop 2026-02-13T02:27:13.842 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.840+0000 7f6f5b9576c0 1 -- 10.20.193.27:0/1327839890 shutdown_connections 2026-02-13T02:27:13.842 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.840+0000 7f6f5b9576c0 1 --2- 10.20.193.27:0/1327839890 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f6f300b5600 0x7f6f300d5ab0 unknown :-1 s=CLOSED pgs=314 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:13.842 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.840+0000 7f6f5b9576c0 1 --2- 10.20.193.27:0/1327839890 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6f54122b50 0x7f6f54122ff0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:13.842 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.840+0000 7f6f5b9576c0 1 --2- 10.20.193.27:0/1327839890 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f6f54115410 0x7f6f5411d790 unknown :-1 s=CLOSED pgs=500 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:13.842 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.840+0000 7f6f5b9576c0 1 --2- 10.20.193.27:0/1327839890 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f6f5410f200 0x7f6f5411d250 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:13.842 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.840+0000 7f6f5b9576c0 1 -- 10.20.193.27:0/1327839890 >> 10.20.193.27:0/1327839890 conn(0x7f6f540ecb00 msgr2=0x7f6f5410cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:27:13.842 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.840+0000 7f6f5b9576c0 1 -- 10.20.193.27:0/1327839890 shutdown_connections 2026-02-13T02:27:13.842 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:13.840+0000 7f6f5b9576c0 1 -- 10.20.193.27:0/1327839890 wait complete. 2026-02-13T02:27:14.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:14 trial028 ceph-mon[26701]: pgmap v339: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:27:14.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:14 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/1327839890' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:14.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:14 trial107 ceph-mon[27422]: pgmap v339: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:27:14.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:14 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/1327839890' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:14.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:14 trial027 ceph-mon[24567]: pgmap v339: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:27:14.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:14 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/1327839890' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:16.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:16 trial028 ceph-mon[26701]: pgmap v340: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:27:16.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:16 trial107 ceph-mon[27422]: pgmap v340: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:27:16.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:16 trial027 ceph-mon[24567]: pgmap v340: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:27:16.883 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:27:17.000 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:27:17.113 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.110+0000 7f28abe806c0 1 -- 10.20.193.27:0/2553220590 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f28a40610b0 msgr2=0x7f28a41550f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:17.113 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.110+0000 7f28abe806c0 1 --2- 10.20.193.27:0/2553220590 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f28a40610b0 0x7f28a41550f0 secure :-1 s=READY pgs=501 gs=1 cs=0 l=1 c_cookie=5576667941222659055 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2898009970 tx=0x7f289802ed20 comp rx=0 tx=0).stop 2026-02-13T02:27:17.113 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.110+0000 7f28abe806c0 1 -- 10.20.193.27:0/2553220590 shutdown_connections 2026-02-13T02:27:17.114 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.110+0000 7f28abe806c0 1 --2- 10.20.193.27:0/2553220590 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f28a4155630 0x7f28a4175a10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:17.114 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.110+0000 7f28abe806c0 1 --2- 10.20.193.27:0/2553220590 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f28a40610b0 0x7f28a41550f0 unknown :-1 s=CLOSED pgs=501 gs=1 cs=0 l=1 c_cookie=5576667941222659055 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:17.114 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.110+0000 7f28abe806c0 1 --2- 10.20.193.27:0/2553220590 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f28a40607a0 0x7f28a4060b70 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:17.114 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.110+0000 7f28abe806c0 1 -- 10.20.193.27:0/2553220590 >> 10.20.193.27:0/2553220590 conn(0x7f28a40ea880 msgr2=0x7f28a410ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:27:17.114 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.110+0000 7f28abe806c0 1 -- 10.20.193.27:0/2553220590 shutdown_connections 2026-02-13T02:27:17.114 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.110+0000 7f28abe806c0 1 -- 10.20.193.27:0/2553220590 wait complete. 2026-02-13T02:27:17.114 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.110+0000 7f28abe806c0 1 Processor -- start 2026-02-13T02:27:17.114 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.110+0000 7f28abe806c0 1 -- start start 2026-02-13T02:27:17.114 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.110+0000 7f28abe806c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f28a40607a0 0x7f28a41614d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:17.114 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.110+0000 7f28abe806c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f28a40610b0 0x7f28a4161a10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:17.114 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.110+0000 7f28abe806c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f28a4155630 0x7f28a4166dd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:17.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.110+0000 7f28abe806c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f28a4167420 con 0x7f28a40610b0 2026-02-13T02:27:17.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.110+0000 7f28abe806c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f28a4167590 con 0x7f28a40607a0 2026-02-13T02:27:17.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.110+0000 7f28abe806c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f28a4167750 con 0x7f28a4155630 2026-02-13T02:27:17.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.110+0000 7f28a93f06c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f28a40610b0 0x7f28a4161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:17.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.110+0000 7f28a93f06c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f28a40610b0 0x7f28a4161a10 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:42044/0 (socket says 10.20.193.27:42044) 2026-02-13T02:27:17.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.110+0000 7f28a93f06c0 1 -- 10.20.193.27:0/4264665473 learned_addr learned my addr 10.20.193.27:0/4264665473 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:27:17.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.110+0000 7f28aa3f26c0 1 --2- 10.20.193.27:0/4264665473 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f28a4155630 0x7f28a4166dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:17.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.110+0000 7f28a9bf16c0 1 --2- 10.20.193.27:0/4264665473 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f28a40607a0 0x7f28a41614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:17.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.111+0000 7f28a93f06c0 1 -- 10.20.193.27:0/4264665473 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f28a4155630 msgr2=0x7f28a4166dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:17.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.111+0000 7f28a93f06c0 1 --2- 10.20.193.27:0/4264665473 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f28a4155630 0x7f28a4166dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:17.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.111+0000 7f28a93f06c0 1 -- 10.20.193.27:0/4264665473 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f28a40607a0 msgr2=0x7f28a41614d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:17.115 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.111+0000 7f28a93f06c0 1 --2- 10.20.193.27:0/4264665473 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f28a40607a0 0x7f28a41614d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:17.116 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.111+0000 7f28a93f06c0 1 -- 10.20.193.27:0/4264665473 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2898009590 con 0x7f28a40610b0 2026-02-13T02:27:17.116 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.111+0000 7f28a93f06c0 1 --2- 10.20.193.27:0/4264665473 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f28a40610b0 0x7f28a4161a10 secure :-1 s=READY pgs=502 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f28980092b0 tx=0x7f289802fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:27:17.116 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.111+0000 7f2892ffd6c0 1 -- 10.20.193.27:0/4264665473 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2898002ba0 con 0x7f28a40610b0 2026-02-13T02:27:17.116 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.111+0000 7f28abe806c0 1 -- 10.20.193.27:0/4264665473 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28a41679d0 con 0x7f28a40610b0 2026-02-13T02:27:17.116 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.111+0000 7f2892ffd6c0 1 -- 10.20.193.27:0/4264665473 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f2898007680 con 0x7f28a40610b0 2026-02-13T02:27:17.116 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.111+0000 7f2892ffd6c0 1 -- 10.20.193.27:0/4264665473 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2898037520 con 0x7f28a40610b0 2026-02-13T02:27:17.116 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.111+0000 7f28abe806c0 1 -- 10.20.193.27:0/4264665473 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28a4151260 con 0x7f28a40610b0 2026-02-13T02:27:17.116 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.111+0000 7f28abe806c0 1 -- 10.20.193.27:0/4264665473 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28a4061cb0 con 0x7f28a40610b0 2026-02-13T02:27:17.117 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.112+0000 7f2892ffd6c0 1 -- 10.20.193.27:0/4264665473 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f2898002d00 con 0x7f28a40610b0 2026-02-13T02:27:17.117 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.113+0000 7f2892ffd6c0 1 --2- 10.20.193.27:0/4264665473 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f28800b5680 0x7f28800d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:17.117 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.113+0000 7f2892ffd6c0 1 -- 10.20.193.27:0/4264665473 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f2898107b00 con 0x7f28a40610b0 2026-02-13T02:27:17.118 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.113+0000 7f28a9bf16c0 1 --2- 10.20.193.27:0/4264665473 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f28800b5680 0x7f28800d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:17.118 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.113+0000 7f28a9bf16c0 1 --2- 10.20.193.27:0/4264665473 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f28800b5680 0x7f28800d5b30 secure :-1 s=READY pgs=315 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2894006b60 tx=0x7f2894006af0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:27:17.118 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.114+0000 7f2892ffd6c0 1 -- 10.20.193.27:0/4264665473 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f28980ca610 con 0x7f28a40610b0 2026-02-13T02:27:17.233 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.231+0000 7f28abe806c0 1 -- 10.20.193.27:0/4264665473 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f28a4060b70 con 0x7f28a40610b0 2026-02-13T02:27:17.233 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.232+0000 7f2892ffd6c0 1 -- 10.20.193.27:0/4264665473 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f28980cd730 con 0x7f28a40610b0 2026-02-13T02:27:17.233 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:27:17.234 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:27:17.235 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.233+0000 7f28abe806c0 1 -- 10.20.193.27:0/4264665473 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f28800b5680 msgr2=0x7f28800d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:17.235 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.233+0000 7f28abe806c0 1 --2- 10.20.193.27:0/4264665473 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f28800b5680 0x7f28800d5b30 secure :-1 s=READY pgs=315 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2894006b60 tx=0x7f2894006af0 comp rx=0 tx=0).stop 2026-02-13T02:27:17.235 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.233+0000 7f28abe806c0 1 -- 10.20.193.27:0/4264665473 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f28a40610b0 msgr2=0x7f28a4161a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:17.235 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.233+0000 7f28abe806c0 1 --2- 10.20.193.27:0/4264665473 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f28a40610b0 0x7f28a4161a10 secure :-1 s=READY pgs=502 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f28980092b0 tx=0x7f289802fd40 comp rx=0 tx=0).stop 2026-02-13T02:27:17.235 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.233+0000 7f28abe806c0 1 -- 10.20.193.27:0/4264665473 shutdown_connections 2026-02-13T02:27:17.235 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.233+0000 7f28abe806c0 1 --2- 10.20.193.27:0/4264665473 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f28800b5680 0x7f28800d5b30 unknown :-1 s=CLOSED pgs=315 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:17.235 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.233+0000 7f28abe806c0 1 --2- 10.20.193.27:0/4264665473 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f28a4155630 0x7f28a4166dd0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:17.235 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.233+0000 7f28abe806c0 1 --2- 10.20.193.27:0/4264665473 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f28a40610b0 0x7f28a4161a10 unknown :-1 s=CLOSED pgs=502 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:17.235 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.233+0000 7f28abe806c0 1 --2- 10.20.193.27:0/4264665473 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f28a40607a0 0x7f28a41614d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:17.235 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.233+0000 7f28abe806c0 1 -- 10.20.193.27:0/4264665473 >> 10.20.193.27:0/4264665473 conn(0x7f28a40ea880 msgr2=0x7f28a410ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:27:17.236 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.233+0000 7f28abe806c0 1 -- 10.20.193.27:0/4264665473 shutdown_connections 2026-02-13T02:27:17.236 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:17.233+0000 7f28abe806c0 1 -- 10.20.193.27:0/4264665473 wait complete. 2026-02-13T02:27:18.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:18 trial028 ceph-mon[26701]: pgmap v341: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:27:18.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:18 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/4264665473' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:18.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:18 trial107 ceph-mon[27422]: pgmap v341: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:27:18.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:18 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/4264665473' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:18.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:18 trial027 ceph-mon[24567]: pgmap v341: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:27:18.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:18 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/4264665473' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:20.275 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:27:20.394 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:27:20.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:20 trial028 ceph-mon[26701]: pgmap v342: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:27:20.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:20 trial107 ceph-mon[27422]: pgmap v342: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:27:20.507 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.504+0000 7f196b6306c0 1 -- 10.20.193.27:0/1114808685 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f196410fc20 msgr2=0x7f1964114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:20.507 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.504+0000 7f196b6306c0 1 --2- 10.20.193.27:0/1114808685 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f196410fc20 0x7f1964114c60 secure :-1 s=READY pgs=503 gs=1 cs=0 l=1 c_cookie=13254549593997658382 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f19580099a0 tx=0x7f195802ed20 comp rx=0 tx=0).stop 2026-02-13T02:27:20.507 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.504+0000 7f196b6306c0 1 -- 10.20.193.27:0/1114808685 shutdown_connections 2026-02-13T02:27:20.507 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.504+0000 7f196b6306c0 1 --2- 10.20.193.27:0/1114808685 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f1964115440 0x7f1964175de0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:20.507 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.504+0000 7f196b6306c0 1 --2- 10.20.193.27:0/1114808685 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f196410fc20 0x7f1964114c60 unknown :-1 s=CLOSED pgs=503 gs=1 cs=0 l=1 c_cookie=13254549593997658382 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:20.507 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.504+0000 7f196b6306c0 1 --2- 10.20.193.27:0/1114808685 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f196410f230 0x7f196410f600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:20.508 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.504+0000 7f196b6306c0 1 -- 10.20.193.27:0/1114808685 >> 10.20.193.27:0/1114808685 conn(0x7f19640ecb30 msgr2=0x7f196410cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:27:20.508 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.504+0000 7f196b6306c0 1 -- 10.20.193.27:0/1114808685 shutdown_connections 2026-02-13T02:27:20.508 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.504+0000 7f196b6306c0 1 -- 10.20.193.27:0/1114808685 wait complete. 2026-02-13T02:27:20.508 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:20 trial027 ceph-mon[24567]: pgmap v342: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.2 KiB/s rd, 2 op/s 2026-02-13T02:27:20.510 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.505+0000 7f196b6306c0 1 Processor -- start 2026-02-13T02:27:20.510 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.505+0000 7f196b6306c0 1 -- start start 2026-02-13T02:27:20.510 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.505+0000 7f196b6306c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f196410f230 0x7f19640666e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:20.510 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.505+0000 7f196b6306c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1964115440 0x7f1964175ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:20.511 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.505+0000 7f196b6306c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1964062940 0x7f1964062de0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:20.511 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.505+0000 7f196b6306c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1964063570 con 0x7f1964062940 2026-02-13T02:27:20.511 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.505+0000 7f196b6306c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f19640636e0 con 0x7f1964115440 2026-02-13T02:27:20.511 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.505+0000 7f196b6306c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f19640ff440 con 0x7f196410f230 2026-02-13T02:27:20.511 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.505+0000 7f1969ba26c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1964062940 0x7f1964062de0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:20.511 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.505+0000 7f1969ba26c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1964062940 0x7f1964062de0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:42058/0 (socket says 10.20.193.27:42058) 2026-02-13T02:27:20.511 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.505+0000 7f1969ba26c0 1 -- 10.20.193.27:0/2847582767 learned_addr learned my addr 10.20.193.27:0/2847582767 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:27:20.511 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.505+0000 7f1968ba06c0 1 --2- 10.20.193.27:0/2847582767 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1964115440 0x7f1964175ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:20.511 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.505+0000 7f19693a16c0 1 --2- 10.20.193.27:0/2847582767 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f196410f230 0x7f19640666e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:20.511 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.505+0000 7f1969ba26c0 1 -- 10.20.193.27:0/2847582767 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f196410f230 msgr2=0x7f19640666e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:20.512 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.505+0000 7f1969ba26c0 1 --2- 10.20.193.27:0/2847582767 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f196410f230 0x7f19640666e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:20.512 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.505+0000 7f1969ba26c0 1 -- 10.20.193.27:0/2847582767 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1964115440 msgr2=0x7f1964175ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:20.512 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.505+0000 7f1969ba26c0 1 --2- 10.20.193.27:0/2847582767 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1964115440 0x7f1964175ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:20.512 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.505+0000 7f1969ba26c0 1 -- 10.20.193.27:0/2847582767 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1958009590 con 0x7f1964062940 2026-02-13T02:27:20.512 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.505+0000 7f19693a16c0 1 --2- 10.20.193.27:0/2847582767 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f196410f230 0x7f19640666e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-13T02:27:20.512 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.505+0000 7f1969ba26c0 1 --2- 10.20.193.27:0/2847582767 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1964062940 0x7f1964062de0 secure :-1 s=READY pgs=504 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1960006ea0 tx=0x7f196000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:27:20.512 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.505+0000 7f19527fc6c0 1 -- 10.20.193.27:0/2847582767 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1960003a30 con 0x7f1964062940 2026-02-13T02:27:20.512 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.505+0000 7f19527fc6c0 1 -- 10.20.193.27:0/2847582767 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f19600054f0 con 0x7f1964062940 2026-02-13T02:27:20.512 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.505+0000 7f196b6306c0 1 -- 10.20.193.27:0/2847582767 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19640ff720 con 0x7f1964062940 2026-02-13T02:27:20.512 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.505+0000 7f19527fc6c0 1 -- 10.20.193.27:0/2847582767 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f196000dcd0 con 0x7f1964062940 2026-02-13T02:27:20.513 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.505+0000 7f196b6306c0 1 -- 10.20.193.27:0/2847582767 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19640ffbb0 con 0x7f1964062940 2026-02-13T02:27:20.513 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.506+0000 7f196b6306c0 1 -- 10.20.193.27:0/2847582767 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1964110500 con 0x7f1964062940 2026-02-13T02:27:20.513 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.507+0000 7f19527fc6c0 1 -- 10.20.193.27:0/2847582767 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f1960005030 con 0x7f1964062940 2026-02-13T02:27:20.513 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.508+0000 7f19527fc6c0 1 --2- 10.20.193.27:0/2847582767 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f19400b5600 0x7f19400d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:20.513 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.508+0000 7f19693a16c0 1 --2- 10.20.193.27:0/2847582767 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f19400b5600 0x7f19400d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:20.514 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.508+0000 7f19527fc6c0 1 -- 10.20.193.27:0/2847582767 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f19600e4b70 con 0x7f1964062940 2026-02-13T02:27:20.514 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.508+0000 7f19693a16c0 1 --2- 10.20.193.27:0/2847582767 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f19400b5600 0x7f19400d5ab0 secure :-1 s=READY pgs=316 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1954004750 tx=0x7f1954009260 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:27:20.514 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.509+0000 7f19527fc6c0 1 -- 10.20.193.27:0/2847582767 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f19600a75f0 con 0x7f1964062940 2026-02-13T02:27:20.627 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.624+0000 7f196b6306c0 1 -- 10.20.193.27:0/2847582767 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f196410f600 con 0x7f1964062940 2026-02-13T02:27:20.627 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.625+0000 7f19527fc6c0 1 -- 10.20.193.27:0/2847582767 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f19600aa7a0 con 0x7f1964062940 2026-02-13T02:27:20.628 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:27:20.628 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:27:20.629 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.626+0000 7f196b6306c0 1 -- 10.20.193.27:0/2847582767 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f19400b5600 msgr2=0x7f19400d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:20.629 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.626+0000 7f196b6306c0 1 --2- 10.20.193.27:0/2847582767 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f19400b5600 0x7f19400d5ab0 secure :-1 s=READY pgs=316 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1954004750 tx=0x7f1954009260 comp rx=0 tx=0).stop 2026-02-13T02:27:20.629 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.626+0000 7f196b6306c0 1 -- 10.20.193.27:0/2847582767 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1964062940 msgr2=0x7f1964062de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:20.629 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.626+0000 7f196b6306c0 1 --2- 10.20.193.27:0/2847582767 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1964062940 0x7f1964062de0 secure :-1 s=READY pgs=504 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1960006ea0 tx=0x7f196000cca0 comp rx=0 tx=0).stop 2026-02-13T02:27:20.629 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.626+0000 7f196b6306c0 1 -- 10.20.193.27:0/2847582767 shutdown_connections 2026-02-13T02:27:20.629 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.626+0000 7f196b6306c0 1 --2- 10.20.193.27:0/2847582767 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f19400b5600 0x7f19400d5ab0 unknown :-1 s=CLOSED pgs=316 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:20.629 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.626+0000 7f196b6306c0 1 --2- 10.20.193.27:0/2847582767 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f1964062940 0x7f1964062de0 unknown :-1 s=CLOSED pgs=504 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:20.629 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.626+0000 7f196b6306c0 1 --2- 10.20.193.27:0/2847582767 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1964115440 0x7f1964175ce0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:20.630 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.626+0000 7f196b6306c0 1 --2- 10.20.193.27:0/2847582767 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f196410f230 0x7f19640666e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:20.630 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.626+0000 7f196b6306c0 1 -- 10.20.193.27:0/2847582767 >> 10.20.193.27:0/2847582767 conn(0x7f19640ecb30 msgr2=0x7f196410d970 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:27:20.630 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.626+0000 7f196b6306c0 1 -- 10.20.193.27:0/2847582767 shutdown_connections 2026-02-13T02:27:20.630 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:20.626+0000 7f196b6306c0 1 -- 10.20.193.27:0/2847582767 wait complete. 2026-02-13T02:27:21.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:21 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2847582767' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:21.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:21 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:27:21.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:21 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:27:21.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:21 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:27:21.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:21 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:27:21.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:21 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:27:21.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:21 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:27:21.494 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:21 trial028 ceph-mon[26701]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:27:21.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:21 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2847582767' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:21.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:21 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:27:21.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:21 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:27:21.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:21 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:27:21.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:21 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:27:21.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:21 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:27:21.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:21 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:27:21.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:21 trial107 ceph-mon[27422]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:27:21.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:21 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2847582767' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:21.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:21 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-13T02:27:21.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:21 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:27:21.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:21 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-13T02:27:21.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:21 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' 2026-02-13T02:27:21.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:21 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-13T02:27:21.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:21 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-13T02:27:21.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:21 trial027 ceph-mon[24567]: from='mgr.14116 10.20.193.27:0/3790080690' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-13T02:27:22.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:22 trial028 ceph-mon[26701]: pgmap v343: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:27:22.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:22 trial028 ceph-mon[26701]: pgmap v344: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 982 B/s rd, 1 op/s 2026-02-13T02:27:22.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:22 trial028 ceph-mon[26701]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:27:22.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:22 trial028 ceph-mon[26701]: Cluster is now healthy 2026-02-13T02:27:22.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:22 trial107 ceph-mon[27422]: pgmap v343: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:27:22.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:22 trial107 ceph-mon[27422]: pgmap v344: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 982 B/s rd, 1 op/s 2026-02-13T02:27:22.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:22 trial107 ceph-mon[27422]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:27:22.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:22 trial107 ceph-mon[27422]: Cluster is now healthy 2026-02-13T02:27:22.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:22 trial027 ceph-mon[24567]: pgmap v343: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 853 B/s rd, 1 op/s 2026-02-13T02:27:22.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:22 trial027 ceph-mon[24567]: pgmap v344: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 982 B/s rd, 1 op/s 2026-02-13T02:27:22.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:22 trial027 ceph-mon[24567]: Health check cleared: CEPHADM_APPLY_SPEC_FAIL (was: Failed to apply 1 service(s): osd.default) 2026-02-13T02:27:22.659 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:22 trial027 ceph-mon[24567]: Cluster is now healthy 2026-02-13T02:27:23.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:23 trial028 ceph-mon[26701]: pgmap v345: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 567 B/s rd, 1 op/s 2026-02-13T02:27:23.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:23 trial028 ceph-mon[26701]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:27:23.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:23 trial107 ceph-mon[27422]: pgmap v345: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 567 B/s rd, 1 op/s 2026-02-13T02:27:23.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:23 trial107 ceph-mon[27422]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:27:23.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:23 trial027 ceph-mon[24567]: pgmap v345: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 567 B/s rd, 1 op/s 2026-02-13T02:27:23.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:23 trial027 ceph-mon[24567]: Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:27:23.670 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:27:23.788 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:27:23.906 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.902+0000 7fd9202216c0 1 -- 10.20.193.27:0/474820774 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd91810f200 msgr2=0x7fd91810f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:23.907 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.902+0000 7fd9202216c0 1 --2- 10.20.193.27:0/474820774 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd91810f200 0x7fd91810f5d0 secure :-1 s=READY pgs=505 gs=1 cs=0 l=1 c_cookie=4279635141085823925 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd9080099a0 tx=0x7fd90802ed20 comp rx=0 tx=0).stop 2026-02-13T02:27:23.907 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.902+0000 7fd9202216c0 1 -- 10.20.193.27:0/474820774 shutdown_connections 2026-02-13T02:27:23.907 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.902+0000 7fd9202216c0 1 --2- 10.20.193.27:0/474820774 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd918115410 0x7fd918175db0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:23.907 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.902+0000 7fd9202216c0 1 --2- 10.20.193.27:0/474820774 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd91810fbf0 0x7fd918114c30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:23.907 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.902+0000 7fd9202216c0 1 --2- 10.20.193.27:0/474820774 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd91810f200 0x7fd91810f5d0 unknown :-1 s=CLOSED pgs=505 gs=1 cs=0 l=1 c_cookie=4279635141085823925 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:23.907 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.902+0000 7fd9202216c0 1 -- 10.20.193.27:0/474820774 >> 10.20.193.27:0/474820774 conn(0x7fd9180ecb00 msgr2=0x7fd91810cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:27:23.907 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.902+0000 7fd9202216c0 1 -- 10.20.193.27:0/474820774 shutdown_connections 2026-02-13T02:27:23.907 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.902+0000 7fd9202216c0 1 -- 10.20.193.27:0/474820774 wait complete. 2026-02-13T02:27:23.907 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.902+0000 7fd9202216c0 1 Processor -- start 2026-02-13T02:27:23.907 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.902+0000 7fd9202216c0 1 -- start start 2026-02-13T02:27:23.907 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.902+0000 7fd9202216c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd91810f200 0x7fd91811d330 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:23.908 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.902+0000 7fd9202216c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd918115410 0x7fd91811d870 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:23.908 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.902+0000 7fd9202216c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd918122c30 0x7fd9181230d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:23.908 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.903+0000 7fd9202216c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd918123640 con 0x7fd91810f200 2026-02-13T02:27:23.908 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.903+0000 7fd9202216c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd9181237b0 con 0x7fd918115410 2026-02-13T02:27:23.908 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.903+0000 7fd9202216c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd918123970 con 0x7fd918122c30 2026-02-13T02:27:23.908 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.903+0000 7fd91df926c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd91810f200 0x7fd91811d330 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:23.908 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.903+0000 7fd91df926c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd91810f200 0x7fd91811d330 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:42078/0 (socket says 10.20.193.27:42078) 2026-02-13T02:27:23.908 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.903+0000 7fd91df926c0 1 -- 10.20.193.27:0/836516021 learned_addr learned my addr 10.20.193.27:0/836516021 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:27:23.908 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.903+0000 7fd91d7916c0 1 --2- 10.20.193.27:0/836516021 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd918115410 0x7fd91811d870 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:23.908 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.903+0000 7fd91df926c0 1 -- 10.20.193.27:0/836516021 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd918122c30 msgr2=0x7fd9181230d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:23.908 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.903+0000 7fd91df926c0 1 --2- 10.20.193.27:0/836516021 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd918122c30 0x7fd9181230d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:23.908 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.903+0000 7fd91df926c0 1 -- 10.20.193.27:0/836516021 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd918115410 msgr2=0x7fd91811d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:23.909 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.903+0000 7fd91df926c0 1 --2- 10.20.193.27:0/836516021 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd918115410 0x7fd91811d870 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:23.909 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.903+0000 7fd91df926c0 1 -- 10.20.193.27:0/836516021 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd908009590 con 0x7fd91810f200 2026-02-13T02:27:23.909 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.903+0000 7fd91df926c0 1 --2- 10.20.193.27:0/836516021 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd91810f200 0x7fd91811d330 secure :-1 s=READY pgs=506 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd90802f230 tx=0x7fd90802fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:27:23.909 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.903+0000 7fd906ffd6c0 1 -- 10.20.193.27:0/836516021 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd908002ae0 con 0x7fd91810f200 2026-02-13T02:27:23.909 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.903+0000 7fd906ffd6c0 1 -- 10.20.193.27:0/836516021 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd908002c40 con 0x7fd91810f200 2026-02-13T02:27:23.909 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.903+0000 7fd9202216c0 1 -- 10.20.193.27:0/836516021 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd918163dc0 con 0x7fd91810f200 2026-02-13T02:27:23.909 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.903+0000 7fd906ffd6c0 1 -- 10.20.193.27:0/836516021 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd9080372f0 con 0x7fd91810f200 2026-02-13T02:27:23.909 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.903+0000 7fd9202216c0 1 -- 10.20.193.27:0/836516021 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd918164240 con 0x7fd91810f200 2026-02-13T02:27:23.909 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.903+0000 7fd9202216c0 1 -- 10.20.193.27:0/836516021 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9181104d0 con 0x7fd91810f200 2026-02-13T02:27:23.910 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.905+0000 7fd906ffd6c0 1 -- 10.20.193.27:0/836516021 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7fd908007810 con 0x7fd91810f200 2026-02-13T02:27:23.911 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.905+0000 7fd906ffd6c0 1 --2- 10.20.193.27:0/836516021 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd8f40b5680 0x7fd8f40d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:23.911 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.905+0000 7fd91d7916c0 1 --2- 10.20.193.27:0/836516021 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd8f40b5680 0x7fd8f40d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:23.911 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.906+0000 7fd906ffd6c0 1 -- 10.20.193.27:0/836516021 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7fd908107d10 con 0x7fd91810f200 2026-02-13T02:27:23.911 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.906+0000 7fd91d7916c0 1 --2- 10.20.193.27:0/836516021 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd8f40b5680 0x7fd8f40d5b30 secure :-1 s=READY pgs=317 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd90c005980 tx=0x7fd90c005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:27:23.911 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:23.906+0000 7fd906ffd6c0 1 -- 10.20.193.27:0/836516021 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7fd9080ce1f0 con 0x7fd91810f200 2026-02-13T02:27:24.023 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:24.020+0000 7fd9202216c0 1 -- 10.20.193.27:0/836516021 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd91810f5d0 con 0x7fd91810f200 2026-02-13T02:27:24.023 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:24.021+0000 7fd906ffd6c0 1 -- 10.20.193.27:0/836516021 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7fd9080cd940 con 0x7fd91810f200 2026-02-13T02:27:24.024 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:27:24.024 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:27:24.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:24.022+0000 7fd9202216c0 1 -- 10.20.193.27:0/836516021 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd8f40b5680 msgr2=0x7fd8f40d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:24.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:24.022+0000 7fd9202216c0 1 --2- 10.20.193.27:0/836516021 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd8f40b5680 0x7fd8f40d5b30 secure :-1 s=READY pgs=317 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd90c005980 tx=0x7fd90c005910 comp rx=0 tx=0).stop 2026-02-13T02:27:24.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:24.022+0000 7fd9202216c0 1 -- 10.20.193.27:0/836516021 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd91810f200 msgr2=0x7fd91811d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:24.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:24.022+0000 7fd9202216c0 1 --2- 10.20.193.27:0/836516021 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd91810f200 0x7fd91811d330 secure :-1 s=READY pgs=506 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd90802f230 tx=0x7fd90802fc00 comp rx=0 tx=0).stop 2026-02-13T02:27:24.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:24.022+0000 7fd9202216c0 1 -- 10.20.193.27:0/836516021 shutdown_connections 2026-02-13T02:27:24.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:24.022+0000 7fd9202216c0 1 --2- 10.20.193.27:0/836516021 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7fd8f40b5680 0x7fd8f40d5b30 unknown :-1 s=CLOSED pgs=317 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:24.025 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:24.022+0000 7fd9202216c0 1 --2- 10.20.193.27:0/836516021 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7fd918122c30 0x7fd9181230d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:24.026 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:24.022+0000 7fd9202216c0 1 --2- 10.20.193.27:0/836516021 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd918115410 0x7fd91811d870 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:24.026 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:24.022+0000 7fd9202216c0 1 --2- 10.20.193.27:0/836516021 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7fd91810f200 0x7fd91811d330 unknown :-1 s=CLOSED pgs=506 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:24.026 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:24.022+0000 7fd9202216c0 1 -- 10.20.193.27:0/836516021 >> 10.20.193.27:0/836516021 conn(0x7fd9180ecb00 msgr2=0x7fd91810cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:27:24.026 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:24.022+0000 7fd9202216c0 1 -- 10.20.193.27:0/836516021 shutdown_connections 2026-02-13T02:27:24.026 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:24.022+0000 7fd9202216c0 1 -- 10.20.193.27:0/836516021 wait complete. 2026-02-13T02:27:24.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:24 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/836516021' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:24.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:24 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/836516021' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:24.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:24 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/836516021' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:25.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:25 trial028 ceph-mon[26701]: pgmap v346: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:27:25.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:25 trial107 ceph-mon[27422]: pgmap v346: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:27:25.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:25 trial027 ceph-mon[24567]: pgmap v346: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:27:27.066 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:27:27.186 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:27:27.306 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.298+0000 7f5428a446c0 1 -- 10.20.193.27:0/2820432457 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f542410fbf0 msgr2=0x7f5424114c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:27.307 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.298+0000 7f5428a446c0 1 --2- 10.20.193.27:0/2820432457 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f542410fbf0 0x7f5424114c30 secure :-1 s=READY pgs=507 gs=1 cs=0 l=1 c_cookie=8650723240396131217 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f540c0099a0 tx=0x7f540c02ed20 comp rx=0 tx=0).stop 2026-02-13T02:27:27.307 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.298+0000 7f5428a446c0 1 -- 10.20.193.27:0/2820432457 shutdown_connections 2026-02-13T02:27:27.307 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.298+0000 7f5428a446c0 1 --2- 10.20.193.27:0/2820432457 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5424115410 0x7f5424175db0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:27.307 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.298+0000 7f5428a446c0 1 --2- 10.20.193.27:0/2820432457 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f542410fbf0 0x7f5424114c30 unknown :-1 s=CLOSED pgs=507 gs=1 cs=0 l=1 c_cookie=8650723240396131217 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:27.307 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.298+0000 7f5428a446c0 1 --2- 10.20.193.27:0/2820432457 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f542410f200 0x7f542410f5d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:27.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.298+0000 7f5428a446c0 1 -- 10.20.193.27:0/2820432457 >> 10.20.193.27:0/2820432457 conn(0x7f54240ecb00 msgr2=0x7f542410cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:27:27.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.299+0000 7f5428a446c0 1 -- 10.20.193.27:0/2820432457 shutdown_connections 2026-02-13T02:27:27.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.299+0000 7f5428a446c0 1 -- 10.20.193.27:0/2820432457 wait complete. 2026-02-13T02:27:27.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.299+0000 7f5428a446c0 1 Processor -- start 2026-02-13T02:27:27.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.299+0000 7f5428a446c0 1 -- start start 2026-02-13T02:27:27.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.299+0000 7f5428a446c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f542410f200 0x7f54240f4c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:27.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.299+0000 7f5428a446c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5424115410 0x7f5424175cb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:27.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.299+0000 7f5428a446c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f54240f52c0 0x7f54240fa590 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:27.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.299+0000 7f5428a446c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f54240f59b0 con 0x7f54240f52c0 2026-02-13T02:27:27.308 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.299+0000 7f5428a446c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f54240faad0 con 0x7f542410f200 2026-02-13T02:27:27.309 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.299+0000 7f5428a446c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f54240fac90 con 0x7f5424115410 2026-02-13T02:27:27.309 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.299+0000 7f5422d766c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f54240f52c0 0x7f54240fa590 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:27.309 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.299+0000 7f5422d766c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f54240f52c0 0x7f54240fa590 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:52682/0 (socket says 10.20.193.27:52682) 2026-02-13T02:27:27.309 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.299+0000 7f5422d766c0 1 -- 10.20.193.27:0/4009743396 learned_addr learned my addr 10.20.193.27:0/4009743396 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:27:27.309 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.299+0000 7f54225756c0 1 --2- 10.20.193.27:0/4009743396 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f542410f200 0x7f54240f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:27.309 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.299+0000 7f5421d746c0 1 --2- 10.20.193.27:0/4009743396 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5424115410 0x7f5424175cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:27.309 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.299+0000 7f5422d766c0 1 -- 10.20.193.27:0/4009743396 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5424115410 msgr2=0x7f5424175cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:27.309 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.299+0000 7f5422d766c0 1 --2- 10.20.193.27:0/4009743396 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5424115410 0x7f5424175cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:27.309 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.299+0000 7f5422d766c0 1 -- 10.20.193.27:0/4009743396 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f542410f200 msgr2=0x7f54240f4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:27.309 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.299+0000 7f5422d766c0 1 --2- 10.20.193.27:0/4009743396 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f542410f200 0x7f54240f4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:27.310 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.299+0000 7f5422d766c0 1 -- 10.20.193.27:0/4009743396 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540c009590 con 0x7f54240f52c0 2026-02-13T02:27:27.310 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.299+0000 7f54225756c0 1 --2- 10.20.193.27:0/4009743396 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f542410f200 0x7f54240f4c90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-13T02:27:27.310 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.299+0000 7f5422d766c0 1 --2- 10.20.193.27:0/4009743396 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f54240f52c0 0x7f54240fa590 secure :-1 s=READY pgs=508 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5414006ea0 tx=0x7f541400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:27:27.310 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.300+0000 7f540b7fe6c0 1 -- 10.20.193.27:0/4009743396 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5414003a30 con 0x7f54240f52c0 2026-02-13T02:27:27.310 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.300+0000 7f5428a446c0 1 -- 10.20.193.27:0/4009743396 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54240faf70 con 0x7f54240f52c0 2026-02-13T02:27:27.310 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.300+0000 7f540b7fe6c0 1 -- 10.20.193.27:0/4009743396 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f54140054f0 con 0x7f54240f52c0 2026-02-13T02:27:27.310 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.300+0000 7f540b7fe6c0 1 -- 10.20.193.27:0/4009743396 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f541400dcd0 con 0x7f54240f52c0 2026-02-13T02:27:27.310 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.300+0000 7f5428a446c0 1 -- 10.20.193.27:0/4009743396 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54240fb700 con 0x7f54240f52c0 2026-02-13T02:27:27.310 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.300+0000 7f5428a446c0 1 -- 10.20.193.27:0/4009743396 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54240fb450 con 0x7f54240f52c0 2026-02-13T02:27:27.310 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.301+0000 7f540b7fe6c0 1 -- 10.20.193.27:0/4009743396 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f5414003b90 con 0x7f54240f52c0 2026-02-13T02:27:27.310 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.302+0000 7f540b7fe6c0 1 --2- 10.20.193.27:0/4009743396 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f53fc0b5600 0x7f53fc0d5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:27.311 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.302+0000 7f54225756c0 1 --2- 10.20.193.27:0/4009743396 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f53fc0b5600 0x7f53fc0d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:27.311 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.302+0000 7f540b7fe6c0 1 -- 10.20.193.27:0/4009743396 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f54140e4b20 con 0x7f54240f52c0 2026-02-13T02:27:27.311 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.302+0000 7f54225756c0 1 --2- 10.20.193.27:0/4009743396 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f53fc0b5600 0x7f53fc0d5ab0 secure :-1 s=READY pgs=318 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5418004750 tx=0x7f5418009260 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:27:27.311 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.302+0000 7f540b7fe6c0 1 -- 10.20.193.27:0/4009743396 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f54140ab000 con 0x7f54240f52c0 2026-02-13T02:27:27.408 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:27 trial027 ceph-mon[24567]: pgmap v347: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:27:27.419 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.416+0000 7f5428a446c0 1 -- 10.20.193.27:0/4009743396 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f54241104d0 con 0x7f54240f52c0 2026-02-13T02:27:27.419 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.416+0000 7f540b7fe6c0 1 -- 10.20.193.27:0/4009743396 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f5414071660 con 0x7f54240f52c0 2026-02-13T02:27:27.419 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:27:27.419 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:27:27.420 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.417+0000 7f5428a446c0 1 -- 10.20.193.27:0/4009743396 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f53fc0b5600 msgr2=0x7f53fc0d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:27.420 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.417+0000 7f5428a446c0 1 --2- 10.20.193.27:0/4009743396 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f53fc0b5600 0x7f53fc0d5ab0 secure :-1 s=READY pgs=318 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5418004750 tx=0x7f5418009260 comp rx=0 tx=0).stop 2026-02-13T02:27:27.420 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.417+0000 7f5428a446c0 1 -- 10.20.193.27:0/4009743396 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f54240f52c0 msgr2=0x7f54240fa590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:27.420 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.417+0000 7f5428a446c0 1 --2- 10.20.193.27:0/4009743396 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f54240f52c0 0x7f54240fa590 secure :-1 s=READY pgs=508 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5414006ea0 tx=0x7f541400cca0 comp rx=0 tx=0).stop 2026-02-13T02:27:27.421 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.417+0000 7f5428a446c0 1 -- 10.20.193.27:0/4009743396 shutdown_connections 2026-02-13T02:27:27.421 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.417+0000 7f5428a446c0 1 --2- 10.20.193.27:0/4009743396 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f53fc0b5600 0x7f53fc0d5ab0 unknown :-1 s=CLOSED pgs=318 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:27.421 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.417+0000 7f5428a446c0 1 --2- 10.20.193.27:0/4009743396 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f54240f52c0 0x7f54240fa590 unknown :-1 s=CLOSED pgs=508 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:27.421 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.417+0000 7f5428a446c0 1 --2- 10.20.193.27:0/4009743396 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f5424115410 0x7f5424175cb0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:27.421 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.417+0000 7f5428a446c0 1 --2- 10.20.193.27:0/4009743396 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f542410f200 0x7f54240f4c90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:27.421 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.417+0000 7f5428a446c0 1 -- 10.20.193.27:0/4009743396 >> 10.20.193.27:0/4009743396 conn(0x7f54240ecb00 msgr2=0x7f5424113d10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:27:27.421 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.417+0000 7f5428a446c0 1 -- 10.20.193.27:0/4009743396 shutdown_connections 2026-02-13T02:27:27.421 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:27.417+0000 7f5428a446c0 1 -- 10.20.193.27:0/4009743396 wait complete. 2026-02-13T02:27:27.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:27 trial028 ceph-mon[26701]: pgmap v347: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:27:27.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:27 trial107 ceph-mon[27422]: pgmap v347: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:27:28.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:28 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/4009743396' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:28.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:28 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/4009743396' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:28.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:28 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/4009743396' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:29.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:29 trial028 ceph-mon[26701]: pgmap v348: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:27:29.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:29 trial107 ceph-mon[27422]: pgmap v348: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:27:29.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:29 trial027 ceph-mon[24567]: pgmap v348: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:27:30.462 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:27:30.578 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:27:30.693 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.689+0000 7faaa565c6c0 1 -- 10.20.193.27:0/261398818 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7faaa0115410 msgr2=0x7faaa0175db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:30.693 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.689+0000 7faaa565c6c0 1 --2- 10.20.193.27:0/261398818 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7faaa0115410 0x7faaa0175db0 secure :-1 s=READY pgs=509 gs=1 cs=0 l=1 c_cookie=9209742600956537053 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7faa9000aa70 tx=0x7faa9002fde0 comp rx=0 tx=0).stop 2026-02-13T02:27:30.694 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.689+0000 7faaa565c6c0 1 -- 10.20.193.27:0/261398818 shutdown_connections 2026-02-13T02:27:30.694 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.689+0000 7faaa565c6c0 1 --2- 10.20.193.27:0/261398818 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7faaa0115410 0x7faaa0175db0 unknown :-1 s=CLOSED pgs=509 gs=1 cs=0 l=1 c_cookie=9209742600956537053 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:30.694 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.689+0000 7faaa565c6c0 1 --2- 10.20.193.27:0/261398818 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7faaa010fbf0 0x7faaa0114c30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:30.694 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.689+0000 7faaa565c6c0 1 --2- 10.20.193.27:0/261398818 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faaa010f200 0x7faaa010f5d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:30.694 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.689+0000 7faaa565c6c0 1 -- 10.20.193.27:0/261398818 >> 10.20.193.27:0/261398818 conn(0x7faaa00ecb00 msgr2=0x7faaa010cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:27:30.694 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.689+0000 7faaa565c6c0 1 -- 10.20.193.27:0/261398818 shutdown_connections 2026-02-13T02:27:30.694 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.689+0000 7faaa565c6c0 1 -- 10.20.193.27:0/261398818 wait complete. 2026-02-13T02:27:30.694 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.689+0000 7faaa565c6c0 1 Processor -- start 2026-02-13T02:27:30.694 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.689+0000 7faaa565c6c0 1 -- start start 2026-02-13T02:27:30.694 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.690+0000 7faaa565c6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7faaa010f200 0x7faaa011d250 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:30.695 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.690+0000 7faaa565c6c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faaa0115410 0x7faaa011d790 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:30.695 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.690+0000 7faaa565c6c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7faaa0122b50 0x7faaa0122ff0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:30.695 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.690+0000 7faaa565c6c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7faaa0123530 con 0x7faaa010f200 2026-02-13T02:27:30.695 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.690+0000 7faaa565c6c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7faaa01236a0 con 0x7faaa0115410 2026-02-13T02:27:30.695 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.690+0000 7faaa565c6c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7faaa0123860 con 0x7faaa0122b50 2026-02-13T02:27:30.695 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.690+0000 7faa9effd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7faaa010f200 0x7faaa011d250 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:30.695 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.690+0000 7faa9effd6c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7faaa010f200 0x7faaa011d250 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:52696/0 (socket says 10.20.193.27:52696) 2026-02-13T02:27:30.695 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.690+0000 7faa9effd6c0 1 -- 10.20.193.27:0/2830950912 learned_addr learned my addr 10.20.193.27:0/2830950912 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:27:30.695 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.690+0000 7faa9effd6c0 1 -- 10.20.193.27:0/2830950912 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7faaa0122b50 msgr2=0x7faaa0122ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:30.695 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.690+0000 7faa9effd6c0 1 --2- 10.20.193.27:0/2830950912 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7faaa0122b50 0x7faaa0122ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:30.695 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.690+0000 7faa9effd6c0 1 -- 10.20.193.27:0/2830950912 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faaa0115410 msgr2=0x7faaa011d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:30.695 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.690+0000 7faa9effd6c0 1 --2- 10.20.193.27:0/2830950912 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faaa0115410 0x7faaa011d790 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:30.696 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.690+0000 7faa9effd6c0 1 -- 10.20.193.27:0/2830950912 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9000a660 con 0x7faaa010f200 2026-02-13T02:27:30.696 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.690+0000 7faa9effd6c0 1 --2- 10.20.193.27:0/2830950912 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7faaa010f200 0x7faaa011d250 secure :-1 s=READY pgs=510 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7faa9400b6c0 tx=0x7faa9400bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:27:30.696 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.690+0000 7faa7ffff6c0 1 -- 10.20.193.27:0/2830950912 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7faa94004560 con 0x7faaa010f200 2026-02-13T02:27:30.696 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.690+0000 7faa7ffff6c0 1 -- 10.20.193.27:0/2830950912 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7faa94009030 con 0x7faaa010f200 2026-02-13T02:27:30.696 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.690+0000 7faaa565c6c0 1 -- 10.20.193.27:0/2830950912 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaa0163ce0 con 0x7faaa010f200 2026-02-13T02:27:30.696 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.690+0000 7faa7ffff6c0 1 -- 10.20.193.27:0/2830950912 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7faa9400cd90 con 0x7faaa010f200 2026-02-13T02:27:30.696 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.690+0000 7faaa565c6c0 1 -- 10.20.193.27:0/2830950912 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaa0164210 con 0x7faaa010f200 2026-02-13T02:27:30.696 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.691+0000 7faaa565c6c0 1 -- 10.20.193.27:0/2830950912 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaa01104d0 con 0x7faaa010f200 2026-02-13T02:27:30.697 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.692+0000 7faa7ffff6c0 1 -- 10.20.193.27:0/2830950912 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7faa940026b0 con 0x7faaa010f200 2026-02-13T02:27:30.697 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.692+0000 7faa7ffff6c0 1 --2- 10.20.193.27:0/2830950912 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7faa740b5680 0x7faa740d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:30.697 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.693+0000 7faa7ffff6c0 1 -- 10.20.193.27:0/2830950912 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7faa940e38c0 con 0x7faaa010f200 2026-02-13T02:27:30.698 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.693+0000 7faa9e7fc6c0 1 --2- 10.20.193.27:0/2830950912 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7faa740b5680 0x7faa740d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:30.698 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.693+0000 7faa9e7fc6c0 1 --2- 10.20.193.27:0/2830950912 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7faa740b5680 0x7faa740d5b30 secure :-1 s=READY pgs=319 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7faa88005980 tx=0x7faa88005910 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:27:30.698 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.693+0000 7faa7ffff6c0 1 -- 10.20.193.27:0/2830950912 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7faa940a6380 con 0x7faaa010f200 2026-02-13T02:27:30.811 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.809+0000 7faaa565c6c0 1 -- 10.20.193.27:0/2830950912 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7faaa010f5d0 con 0x7faaa010f200 2026-02-13T02:27:30.811 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.809+0000 7faa7ffff6c0 1 -- 10.20.193.27:0/2830950912 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7faa940a9530 con 0x7faaa010f200 2026-02-13T02:27:30.812 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:27:30.812 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:27:30.813 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.810+0000 7faaa565c6c0 1 -- 10.20.193.27:0/2830950912 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7faa740b5680 msgr2=0x7faa740d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:30.813 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.810+0000 7faaa565c6c0 1 --2- 10.20.193.27:0/2830950912 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7faa740b5680 0x7faa740d5b30 secure :-1 s=READY pgs=319 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7faa88005980 tx=0x7faa88005910 comp rx=0 tx=0).stop 2026-02-13T02:27:30.813 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.810+0000 7faaa565c6c0 1 -- 10.20.193.27:0/2830950912 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7faaa010f200 msgr2=0x7faaa011d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:30.813 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.810+0000 7faaa565c6c0 1 --2- 10.20.193.27:0/2830950912 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7faaa010f200 0x7faaa011d250 secure :-1 s=READY pgs=510 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7faa9400b6c0 tx=0x7faa9400bb90 comp rx=0 tx=0).stop 2026-02-13T02:27:30.813 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.810+0000 7faaa565c6c0 1 -- 10.20.193.27:0/2830950912 shutdown_connections 2026-02-13T02:27:30.813 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.810+0000 7faaa565c6c0 1 --2- 10.20.193.27:0/2830950912 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7faa740b5680 0x7faa740d5b30 unknown :-1 s=CLOSED pgs=319 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:30.813 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.810+0000 7faaa565c6c0 1 --2- 10.20.193.27:0/2830950912 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7faaa0122b50 0x7faaa0122ff0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:30.813 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.810+0000 7faaa565c6c0 1 --2- 10.20.193.27:0/2830950912 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faaa0115410 0x7faaa011d790 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:30.813 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.810+0000 7faaa565c6c0 1 --2- 10.20.193.27:0/2830950912 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7faaa010f200 0x7faaa011d250 unknown :-1 s=CLOSED pgs=510 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:30.813 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.810+0000 7faaa565c6c0 1 -- 10.20.193.27:0/2830950912 >> 10.20.193.27:0/2830950912 conn(0x7faaa00ecb00 msgr2=0x7faaa010cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:27:30.814 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.810+0000 7faaa565c6c0 1 -- 10.20.193.27:0/2830950912 shutdown_connections 2026-02-13T02:27:30.814 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:30.810+0000 7faaa565c6c0 1 -- 10.20.193.27:0/2830950912 wait complete. 2026-02-13T02:27:31.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:31 trial028 ceph-mon[26701]: pgmap v349: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:27:31.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:31 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/2830950912' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:31.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:31 trial107 ceph-mon[27422]: pgmap v349: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:27:31.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:31 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/2830950912' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:31.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:31 trial027 ceph-mon[24567]: pgmap v349: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 1.1 KiB/s rd, 2 op/s 2026-02-13T02:27:31.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:31 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/2830950912' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:33.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:33 trial028 ceph-mon[26701]: pgmap v350: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 966 B/s rd, 1 op/s 2026-02-13T02:27:33.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:33 trial107 ceph-mon[27422]: pgmap v350: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 966 B/s rd, 1 op/s 2026-02-13T02:27:33.658 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:33 trial027 ceph-mon[24567]: pgmap v350: 97 pgs: 97 active+clean; 647 KiB data, 325 MiB used, 8.2 TiB / 8.2 TiB avail; 966 B/s rd, 1 op/s 2026-02-13T02:27:33.851 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph health --format=json 2026-02-13T02:27:33.970 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:27:34.088 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.083+0000 7f62b38b36c0 1 -- 10.20.193.27:0/3004843977 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f62ac10ba80 msgr2=0x7f62ac10be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:34.088 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.083+0000 7f62b38b36c0 1 --2- 10.20.193.27:0/3004843977 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f62ac10ba80 0x7f62ac10be50 secure :-1 s=READY pgs=511 gs=1 cs=0 l=1 c_cookie=10315952732813985805 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f629c009970 tx=0x7f629c02ed20 comp rx=0 tx=0).stop 2026-02-13T02:27:34.088 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.083+0000 7f62b38b36c0 1 -- 10.20.193.27:0/3004843977 shutdown_connections 2026-02-13T02:27:34.088 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.083+0000 7f62b38b36c0 1 --2- 10.20.193.27:0/3004843977 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f62ac1557b0 0x7f62ac175b90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:34.088 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.083+0000 7f62b38b36c0 1 --2- 10.20.193.27:0/3004843977 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f62ac10c390 0x7f62ac155270 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:34.088 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.083+0000 7f62b38b36c0 1 --2- 10.20.193.27:0/3004843977 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f62ac10ba80 0x7f62ac10be50 unknown :-1 s=CLOSED pgs=511 gs=1 cs=0 l=1 c_cookie=10315952732813985805 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:34.088 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.083+0000 7f62b38b36c0 1 -- 10.20.193.27:0/3004843977 >> 10.20.193.27:0/3004843977 conn(0x7f62ac0e9740 msgr2=0x7f62ac109b50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:27:34.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.083+0000 7f62b38b36c0 1 -- 10.20.193.27:0/3004843977 shutdown_connections 2026-02-13T02:27:34.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.084+0000 7f62b38b36c0 1 -- 10.20.193.27:0/3004843977 wait complete. 2026-02-13T02:27:34.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.084+0000 7f62b38b36c0 1 Processor -- start 2026-02-13T02:27:34.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.084+0000 7f62b38b36c0 1 -- start start 2026-02-13T02:27:34.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.084+0000 7f62b38b36c0 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f62ac10ba80 0x7f62ac160640 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:34.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.084+0000 7f62b38b36c0 1 --2- >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f62ac10c390 0x7f62ac160b80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:34.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.084+0000 7f62b38b36c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f62ac1557b0 0x7f62ac165f40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:34.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.084+0000 7f62b38b36c0 1 -- --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac166590 con 0x7f62ac1557b0 2026-02-13T02:27:34.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.084+0000 7f62b38b36c0 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac166700 con 0x7f62ac10ba80 2026-02-13T02:27:34.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.084+0000 7f62b38b36c0 1 -- --> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac1668c0 con 0x7f62ac10c390 2026-02-13T02:27:34.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.084+0000 7f62b1e256c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f62ac1557b0 0x7f62ac165f40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:34.089 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.084+0000 7f62b1e256c0 1 --2- >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f62ac1557b0 0x7f62ac165f40 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.27:3300/0 says I am v2:10.20.193.27:52710/0 (socket says 10.20.193.27:52710) 2026-02-13T02:27:34.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.084+0000 7f62b1e256c0 1 -- 10.20.193.27:0/891960082 learned_addr learned my addr 10.20.193.27:0/891960082 (peer_addr_for_me v2:10.20.193.27:0/0) 2026-02-13T02:27:34.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.084+0000 7f62b0e236c0 1 --2- 10.20.193.27:0/891960082 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f62ac10c390 0x7f62ac160b80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:34.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.084+0000 7f62b16246c0 1 --2- 10.20.193.27:0/891960082 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f62ac10ba80 0x7f62ac160640 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:34.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.084+0000 7f62b1e256c0 1 -- 10.20.193.27:0/891960082 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f62ac10c390 msgr2=0x7f62ac160b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:34.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.084+0000 7f62b1e256c0 1 --2- 10.20.193.27:0/891960082 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f62ac10c390 0x7f62ac160b80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:34.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.084+0000 7f62b1e256c0 1 -- 10.20.193.27:0/891960082 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f62ac10ba80 msgr2=0x7f62ac160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:34.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.084+0000 7f62b1e256c0 1 --2- 10.20.193.27:0/891960082 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f62ac10ba80 0x7f62ac160640 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:34.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.084+0000 7f62b1e256c0 1 -- 10.20.193.27:0/891960082 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f629c009590 con 0x7f62ac1557b0 2026-02-13T02:27:34.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.084+0000 7f62b1e256c0 1 --2- 10.20.193.27:0/891960082 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f62ac1557b0 0x7f62ac165f40 secure :-1 s=READY pgs=512 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f62a8006ea0 tx=0x7f62a800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:27:34.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.084+0000 7f629a7fc6c0 1 -- 10.20.193.27:0/891960082 <== mon.0 v2:10.20.193.27:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f62a8003a30 con 0x7f62ac1557b0 2026-02-13T02:27:34.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.085+0000 7f629a7fc6c0 1 -- 10.20.193.27:0/891960082 <== mon.0 v2:10.20.193.27:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f62a80054f0 con 0x7f62ac1557b0 2026-02-13T02:27:34.090 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.085+0000 7f629a7fc6c0 1 -- 10.20.193.27:0/891960082 <== mon.0 v2:10.20.193.27:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f62a800dcd0 con 0x7f62ac1557b0 2026-02-13T02:27:34.091 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.085+0000 7f62b38b36c0 1 -- 10.20.193.27:0/891960082 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62ac166ba0 con 0x7f62ac1557b0 2026-02-13T02:27:34.091 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.085+0000 7f62b38b36c0 1 -- 10.20.193.27:0/891960082 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62ac1523f0 con 0x7f62ac1557b0 2026-02-13T02:27:34.091 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.085+0000 7f62b38b36c0 1 -- 10.20.193.27:0/891960082 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62ac166dc0 con 0x7f62ac1557b0 2026-02-13T02:27:34.092 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.087+0000 7f629a7fc6c0 1 -- 10.20.193.27:0/891960082 <== mon.0 v2:10.20.193.27:3300/0 4 ==== mgrmap(e 13) ==== 157066+0+0 (secure 0 0 0) 0x7f62a8005030 con 0x7f62ac1557b0 2026-02-13T02:27:34.092 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.087+0000 7f629a7fc6c0 1 --2- 10.20.193.27:0/891960082 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f62880b5680 0x7f62880d5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-13T02:27:34.092 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.087+0000 7f62b16246c0 1 --2- 10.20.193.27:0/891960082 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f62880b5680 0x7f62880d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-13T02:27:34.092 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.088+0000 7f629a7fc6c0 1 -- 10.20.193.27:0/891960082 <== mon.0 v2:10.20.193.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8031+0+0 (secure 0 0 0) 0x7f62a80e4ac0 con 0x7f62ac1557b0 2026-02-13T02:27:34.092 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.088+0000 7f62b16246c0 1 --2- 10.20.193.27:0/891960082 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f62880b5680 0x7f62880d5b30 secure :-1 s=READY pgs=320 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f629c005b40 tx=0x7f629c005c00 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-13T02:27:34.092 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.088+0000 7f629a7fc6c0 1 -- 10.20.193.27:0/891960082 <== mon.0 v2:10.20.193.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+230863 (secure 0 0 0) 0x7f62a80aa730 con 0x7f62ac1557b0 2026-02-13T02:27:34.205 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.203+0000 7f62b38b36c0 1 -- 10.20.193.27:0/891960082 --> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f62ac161790 con 0x7f62ac1557b0 2026-02-13T02:27:34.206 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.203+0000 7f629a7fc6c0 1 -- 10.20.193.27:0/891960082 <== mon.0 v2:10.20.193.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+190 (secure 0 0 0) 0x7f62a80aa910 con 0x7f62ac1557b0 2026-02-13T02:27:34.206 INFO:teuthology.orchestra.run.trial027.stdout: 2026-02-13T02:27:34.206 INFO:teuthology.orchestra.run.trial027.stdout:{"status":"HEALTH_WARN","checks":{"CEPHADM_APPLY_SPEC_FAIL":{"severity":"HEALTH_WARN","summary":{"message":"Failed to apply 1 service(s): osd.default","count":1},"muted":false}},"mutes":[]} 2026-02-13T02:27:34.207 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.204+0000 7f62b38b36c0 1 -- 10.20.193.27:0/891960082 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f62880b5680 msgr2=0x7f62880d5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:34.207 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.204+0000 7f62b38b36c0 1 --2- 10.20.193.27:0/891960082 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f62880b5680 0x7f62880d5b30 secure :-1 s=READY pgs=320 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f629c005b40 tx=0x7f629c005c00 comp rx=0 tx=0).stop 2026-02-13T02:27:34.207 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.204+0000 7f62b38b36c0 1 -- 10.20.193.27:0/891960082 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f62ac1557b0 msgr2=0x7f62ac165f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-13T02:27:34.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.204+0000 7f62b38b36c0 1 --2- 10.20.193.27:0/891960082 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f62ac1557b0 0x7f62ac165f40 secure :-1 s=READY pgs=512 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f62a8006ea0 tx=0x7f62a800cca0 comp rx=0 tx=0).stop 2026-02-13T02:27:34.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.204+0000 7f62b38b36c0 1 -- 10.20.193.27:0/891960082 shutdown_connections 2026-02-13T02:27:34.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.204+0000 7f62b38b36c0 1 --2- 10.20.193.27:0/891960082 >> [v2:10.20.193.27:6800/1837452956,v1:10.20.193.27:6801/1837452956] conn(0x7f62880b5680 0x7f62880d5b30 unknown :-1 s=CLOSED pgs=320 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:34.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.204+0000 7f62b38b36c0 1 --2- 10.20.193.27:0/891960082 >> [v2:10.20.193.27:3300/0,v1:10.20.193.27:6789/0] conn(0x7f62ac1557b0 0x7f62ac165f40 unknown :-1 s=CLOSED pgs=512 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:34.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.204+0000 7f62b38b36c0 1 --2- 10.20.193.27:0/891960082 >> [v2:10.20.193.107:3300/0,v1:10.20.193.107:6789/0] conn(0x7f62ac10c390 0x7f62ac160b80 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:34.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.204+0000 7f62b38b36c0 1 --2- 10.20.193.27:0/891960082 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f62ac10ba80 0x7f62ac160640 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-13T02:27:34.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.204+0000 7f62b38b36c0 1 -- 10.20.193.27:0/891960082 >> 10.20.193.27:0/891960082 conn(0x7f62ac0e9740 msgr2=0x7f62ac109b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-13T02:27:34.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.204+0000 7f62b38b36c0 1 -- 10.20.193.27:0/891960082 shutdown_connections 2026-02-13T02:27:34.208 INFO:teuthology.orchestra.run.trial027.stderr:2026-02-13T02:27:34.204+0000 7f62b38b36c0 1 -- 10.20.193.27:0/891960082 wait complete. 2026-02-13T02:27:34.249 INFO:tasks.cephadm:Teardown begin 2026-02-13T02:27:34.249 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_a2c505dae37458c5537fa44f1320bd04eef25bb1/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_batrick_ceph_cfcf307502511ba51c39a8b9085c2fcde505d421/qa/tasks/cephadm.py", line 2025, in task healthy(ctx=ctx, config=config) File "/home/teuthworker/src/github.com_batrick_ceph_cfcf307502511ba51c39a8b9085c2fcde505d421/qa/tasks/ceph.py", line 1557, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/github.com_batrick_ceph_cfcf307502511ba51c39a8b9085c2fcde505d421/qa/tasks/ceph_manager.py", line 3393, in wait_until_healthy assert time.time() - start < timeout, \ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AssertionError: timeout expired in wait_until_healthy 2026-02-13T02:27:34.253 DEBUG:teuthology.orchestra.run.trial027:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2026-02-13T02:27:34.280 DEBUG:teuthology.orchestra.run.trial028:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2026-02-13T02:27:34.308 DEBUG:teuthology.orchestra.run.trial107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2026-02-13T02:27:34.337 INFO:tasks.cephadm:Disabling cephadm mgr module 2026-02-13T02:27:34.337 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:8f8d38294e4db8750b4b1649f62ce37caa63080f-rockylinux-10 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 -- ceph mgr module disable cephadm 2026-02-13T02:27:34.452 INFO:teuthology.orchestra.run.trial027.stderr:Inferring config /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/mon.a/config 2026-02-13T02:27:34.469 INFO:teuthology.orchestra.run.trial027.stderr:Error: statfs /etc/ceph/ceph.client.admin.keyring: no such file or directory 2026-02-13T02:27:34.470 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:34 trial027 ceph-mon[24567]: from='client.? 10.20.193.27:0/891960082' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:34.484 DEBUG:teuthology.orchestra.run:got remote process result: 125 2026-02-13T02:27:34.485 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2026-02-13T02:27:34.485 DEBUG:teuthology.orchestra.run.trial027:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2026-02-13T02:27:34.493 INFO:journalctl@ceph.mon.b.trial028.stdout:Feb 13 02:27:34 trial028 ceph-mon[26701]: from='client.? 10.20.193.27:0/891960082' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:34.501 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:34 trial107 ceph-mon[27422]: from='client.? 10.20.193.27:0/891960082' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-13T02:27:34.502 DEBUG:teuthology.orchestra.run.trial028:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2026-02-13T02:27:34.518 DEBUG:teuthology.orchestra.run.trial107:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2026-02-13T02:27:34.535 INFO:tasks.cephadm:Stopping all daemons... 2026-02-13T02:27:34.535 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2026-02-13T02:27:34.536 DEBUG:teuthology.orchestra.run.trial027:> sudo systemctl stop ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mon.a 2026-02-13T02:27:34.830 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:34 trial027 systemd[1]: Stopping Ceph mon.a for 707dbb29-0882-11f1-9262-d404e6e7d460... 2026-02-13T02:27:34.830 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:34 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mon-a[24560]: 2026-02-13T02:27:34.645+0000 7f18140896c0 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2026-02-13T02:27:34.830 INFO:journalctl@ceph.mon.a.trial027.stdout:Feb 13 02:27:34 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mon-a[24560]: 2026-02-13T02:27:34.645+0000 7f18140896c0 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2026-02-13T02:27:34.914 DEBUG:teuthology.orchestra.run.trial027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mon.a.service' 2026-02-13T02:27:34.952 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T02:27:34.952 INFO:tasks.cephadm.mon.a:Stopped mon.a 2026-02-13T02:27:34.952 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2026-02-13T02:27:34.953 DEBUG:teuthology.orchestra.run.trial028:> sudo systemctl stop ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mon.b 2026-02-13T02:27:35.160 DEBUG:teuthology.orchestra.run.trial028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mon.b.service' 2026-02-13T02:27:35.198 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T02:27:35.199 INFO:tasks.cephadm.mon.c:Stopped mon.b 2026-02-13T02:27:35.199 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2026-02-13T02:27:35.199 DEBUG:teuthology.orchestra.run.trial107:> sudo systemctl stop ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mon.c 2026-02-13T02:27:35.490 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:35 trial107 systemd[1]: Stopping Ceph mon.c for 707dbb29-0882-11f1-9262-d404e6e7d460... 2026-02-13T02:27:35.491 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:35 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mon-c[27414]: 2026-02-13T02:27:35.300+0000 7f2e143df6c0 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2026-02-13T02:27:35.491 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:35 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mon-c[27414]: 2026-02-13T02:27:35.300+0000 7f2e143df6c0 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2026-02-13T02:27:35.491 INFO:journalctl@ceph.mon.c.trial107.stdout:Feb 13 02:27:35 trial107 podman[59851]: 2026-02-13 02:27:35.493242072 +0000 UTC m=+0.205243917 container died e2fb8c60b0d0e04fd54194cf74866c57f5fe8c80d0704e96090bab2e2034f000 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mon-c, org.opencontainers.image.authors=Ceph Release Team , FROM_IMAGE=docker.io/rockylinux/rockylinux:10, io.buildah.version=1.33.7, org.opencontainers.image.version=10, org.opencontainers.image.licenses=BSD-3-Clause, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.title=rockylinux, OSD_FLAVOR=default, vendor=Rocky Enterprise Software Foundation, name=rockylinux, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, version=10, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, ceph=True, CEPH_REF=wip-pdonnell-testing-20260210.171210, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, org.opencontainers.image.documentation=https://docs.ceph.com/, license=BSD-3-Clause, summary=Rocky Linux Base image, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git) 2026-02-13T02:27:35.567 DEBUG:teuthology.orchestra.run.trial107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mon.c.service' 2026-02-13T02:27:35.606 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T02:27:35.606 INFO:tasks.cephadm.mon.c:Stopped mon.c 2026-02-13T02:27:35.606 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2026-02-13T02:27:35.606 DEBUG:teuthology.orchestra.run.trial027:> sudo systemctl stop ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mgr.x 2026-02-13T02:27:35.840 DEBUG:teuthology.orchestra.run.trial027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mgr.x.service' 2026-02-13T02:27:35.875 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T02:27:35.875 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2026-02-13T02:27:35.875 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2026-02-13T02:27:35.876 DEBUG:teuthology.orchestra.run.trial028:> sudo systemctl stop ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mgr.y 2026-02-13T02:27:35.983 INFO:journalctl@ceph.mgr.y.trial028.stdout:Feb 13 02:27:35 trial028 systemd[1]: Stopping Ceph mgr.y for 707dbb29-0882-11f1-9262-d404e6e7d460... 2026-02-13T02:27:36.075 DEBUG:teuthology.orchestra.run.trial028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mgr.y.service' 2026-02-13T02:27:36.111 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T02:27:36.111 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2026-02-13T02:27:36.111 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2026-02-13T02:27:36.112 DEBUG:teuthology.orchestra.run.trial107:> sudo systemctl stop ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mgr.z 2026-02-13T02:27:36.300 DEBUG:teuthology.orchestra.run.trial107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mgr.z.service' 2026-02-13T02:27:36.335 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T02:27:36.335 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2026-02-13T02:27:36.335 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2026-02-13T02:27:36.335 DEBUG:teuthology.orchestra.run.trial027:> sudo systemctl stop ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.0 2026-02-13T02:27:36.658 INFO:journalctl@ceph.osd.0.trial027.stdout:Feb 13 02:27:36 trial027 systemd[1]: Stopping Ceph osd.0 for 707dbb29-0882-11f1-9262-d404e6e7d460... 2026-02-13T02:27:36.658 INFO:journalctl@ceph.osd.0.trial027.stdout:Feb 13 02:27:36 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-0[36849]: 2026-02-13T02:27:36.428+0000 7f8499b206c0 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --osd-objectstore=bluestore (PID: 1) UID: 0 2026-02-13T02:27:36.659 INFO:journalctl@ceph.osd.0.trial027.stdout:Feb 13 02:27:36 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-0[36849]: 2026-02-13T02:27:36.428+0000 7f8499b206c0 -1 osd.0 72 *** Got signal Terminated *** 2026-02-13T02:27:36.659 INFO:journalctl@ceph.osd.0.trial027.stdout:Feb 13 02:27:36 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-0[36849]: 2026-02-13T02:27:36.428+0000 7f8499b206c0 -1 osd.0 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-13T02:27:41.793 INFO:journalctl@ceph.osd.0.trial027.stdout:Feb 13 02:27:41 trial027 podman[81344]: 2026-02-13 02:27:41.49959015 +0000 UTC m=+5.081733269 container died e8f4b7eb689f5152c0d6cd0e8c410cf93f736e5df408e276b0d93f72c551d81f (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-0, ceph=True, CEPH_REF=wip-pdonnell-testing-20260210.171210, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, summary=Rocky Linux Base image, org.opencontainers.image.authors=Ceph Release Team , CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, name=rockylinux, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, version=10, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, io.buildah.version=1.33.7, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.version=10, vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.title=rockylinux, OSD_FLAVOR=default, org.opencontainers.image.licenses=BSD-3-Clause, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, license=BSD-3-Clause) 2026-02-13T02:27:41.793 INFO:journalctl@ceph.osd.0.trial027.stdout:Feb 13 02:27:41 trial027 podman[81344]: 2026-02-13 02:27:41.507739028 +0000 UTC m=+5.089882147 container remove e8f4b7eb689f5152c0d6cd0e8c410cf93f736e5df408e276b0d93f72c551d81f (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-0, org.opencontainers.image.authors=Ceph Release Team , version=10, CEPH_REF=wip-pdonnell-testing-20260210.171210, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, vendor=Rocky Enterprise Software Foundation, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, ceph=True, org.opencontainers.image.title=rockylinux, org.opencontainers.image.licenses=BSD-3-Clause, OSD_FLAVOR=default, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, org.opencontainers.image.version=10, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, io.buildah.version=1.33.7, name=rockylinux, license=BSD-3-Clause, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, summary=Rocky Linux Base image, org.opencontainers.image.documentation=https://docs.ceph.com/) 2026-02-13T02:27:41.793 INFO:journalctl@ceph.osd.0.trial027.stdout:Feb 13 02:27:41 trial027 bash[81344]: ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-0 2026-02-13T02:27:41.793 INFO:journalctl@ceph.osd.0.trial027.stdout:Feb 13 02:27:41 trial027 podman[81477]: 2026-02-13 02:27:41.631326205 +0000 UTC m=+0.014956047 container create 09fe84f532ed7d1649de43df13b198c3483767b1e744896d7dd8bcbd2d5ab962 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-0-deactivate, version=10, org.opencontainers.image.title=rockylinux, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, summary=Rocky Linux Base image, license=BSD-3-Clause, CEPH_REF=wip-pdonnell-testing-20260210.171210, OSD_FLAVOR=default, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.licenses=BSD-3-Clause, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, vendor=Rocky Enterprise Software Foundation, io.buildah.version=1.33.7, name=rockylinux, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.version=10, ceph=True, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation) 2026-02-13T02:27:41.793 INFO:journalctl@ceph.osd.0.trial027.stdout:Feb 13 02:27:41 trial027 podman[81477]: 2026-02-13 02:27:41.667920481 +0000 UTC m=+0.051550333 container init 09fe84f532ed7d1649de43df13b198c3483767b1e744896d7dd8bcbd2d5ab962 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-0-deactivate, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, CEPH_REF=wip-pdonnell-testing-20260210.171210, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.version=10, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.documentation=https://docs.ceph.com/, version=10, org.opencontainers.image.licenses=BSD-3-Clause, org.opencontainers.image.title=rockylinux, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, summary=Rocky Linux Base image, OSD_FLAVOR=default, org.opencontainers.image.authors=Ceph Release Team , ceph=True, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, vendor=Rocky Enterprise Software Foundation, license=BSD-3-Clause, io.buildah.version=1.33.7, name=rockylinux) 2026-02-13T02:27:41.793 INFO:journalctl@ceph.osd.0.trial027.stdout:Feb 13 02:27:41 trial027 podman[81477]: 2026-02-13 02:27:41.670428216 +0000 UTC m=+0.054058058 container start 09fe84f532ed7d1649de43df13b198c3483767b1e744896d7dd8bcbd2d5ab962 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-0-deactivate, license=BSD-3-Clause, name=rockylinux, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, CEPH_REF=wip-pdonnell-testing-20260210.171210, summary=Rocky Linux Base image, org.opencontainers.image.authors=Ceph Release Team , io.buildah.version=1.33.7, version=10, OSD_FLAVOR=default, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, ceph=True, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, org.opencontainers.image.title=rockylinux, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, org.opencontainers.image.licenses=BSD-3-Clause, org.opencontainers.image.version=10, vendor=Rocky Enterprise Software Foundation) 2026-02-13T02:27:41.794 INFO:journalctl@ceph.osd.0.trial027.stdout:Feb 13 02:27:41 trial027 podman[81477]: 2026-02-13 02:27:41.67087179 +0000 UTC m=+0.054501622 container attach 09fe84f532ed7d1649de43df13b198c3483767b1e744896d7dd8bcbd2d5ab962 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-0-deactivate, CEPH_REF=wip-pdonnell-testing-20260210.171210, org.opencontainers.image.authors=Ceph Release Team , GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, license=BSD-3-Clause, version=10, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, vendor=Rocky Enterprise Software Foundation, io.buildah.version=1.33.7, name=rockylinux, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, OSD_FLAVOR=default, org.opencontainers.image.version=10, org.opencontainers.image.licenses=BSD-3-Clause, org.opencontainers.image.title=rockylinux, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, summary=Rocky Linux Base image, ceph=True) 2026-02-13T02:27:41.794 INFO:journalctl@ceph.osd.0.trial027.stdout:Feb 13 02:27:41 trial027 podman[81477]: 2026-02-13 02:27:41.62588348 +0000 UTC m=+0.009513332 image pull b86a4a15c2dcd78db08124ee64cb48fcd681d6db032e313da19d063813b235da quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9 2026-02-13T02:27:41.794 INFO:journalctl@ceph.osd.0.trial027.stdout:Feb 13 02:27:41 trial027 podman[81477]: 2026-02-13 02:27:41.791517415 +0000 UTC m=+0.175147257 container died 09fe84f532ed7d1649de43df13b198c3483767b1e744896d7dd8bcbd2d5ab962 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-0-deactivate, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, ceph=True, name=rockylinux, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, CEPH_REF=wip-pdonnell-testing-20260210.171210, summary=Rocky Linux Base image, OSD_FLAVOR=default, vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.licenses=BSD-3-Clause, io.buildah.version=1.33.7, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.version=10, version=10, org.opencontainers.image.title=rockylinux, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, license=BSD-3-Clause, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git) 2026-02-13T02:27:41.820 DEBUG:teuthology.orchestra.run.trial027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.0.service' 2026-02-13T02:27:41.856 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T02:27:41.857 INFO:tasks.cephadm.osd.0:Stopped osd.0 2026-02-13T02:27:41.857 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2026-02-13T02:27:41.857 DEBUG:teuthology.orchestra.run.trial028:> sudo systemctl stop ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.1 2026-02-13T02:27:42.243 INFO:journalctl@ceph.osd.1.trial028.stdout:Feb 13 02:27:41 trial028 systemd[1]: Stopping Ceph osd.1 for 707dbb29-0882-11f1-9262-d404e6e7d460... 2026-02-13T02:27:42.244 INFO:journalctl@ceph.osd.1.trial028.stdout:Feb 13 02:27:41 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-1[31494]: 2026-02-13T02:27:41.957+0000 7fe0f414b6c0 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --osd-objectstore=bluestore (PID: 1) UID: 0 2026-02-13T02:27:42.244 INFO:journalctl@ceph.osd.1.trial028.stdout:Feb 13 02:27:41 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-1[31494]: 2026-02-13T02:27:41.957+0000 7fe0f414b6c0 -1 osd.1 72 *** Got signal Terminated *** 2026-02-13T02:27:42.244 INFO:journalctl@ceph.osd.1.trial028.stdout:Feb 13 02:27:41 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-1[31494]: 2026-02-13T02:27:41.957+0000 7fe0f414b6c0 -1 osd.1 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-13T02:27:47.320 INFO:journalctl@ceph.osd.1.trial028.stdout:Feb 13 02:27:47 trial028 podman[57584]: 2026-02-13 02:27:47.026635429 +0000 UTC m=+5.081456962 container died 6b181145649cfce26f619434c76284692a117820a0b19e8d32c27b98b3c0e65f (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-1, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.licenses=BSD-3-Clause, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.version=10, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, ceph=True, org.opencontainers.image.title=rockylinux, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, license=BSD-3-Clause, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, name=rockylinux, CEPH_REF=wip-pdonnell-testing-20260210.171210, version=10, io.buildah.version=1.33.7, summary=Rocky Linux Base image, OSD_FLAVOR=default, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, vendor=Rocky Enterprise Software Foundation, FROM_IMAGE=docker.io/rockylinux/rockylinux:10) 2026-02-13T02:27:47.320 INFO:journalctl@ceph.osd.1.trial028.stdout:Feb 13 02:27:47 trial028 podman[57584]: 2026-02-13 02:27:47.035810835 +0000 UTC m=+5.090632368 container remove 6b181145649cfce26f619434c76284692a117820a0b19e8d32c27b98b3c0e65f (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-1, org.opencontainers.image.title=rockylinux, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, vendor=Rocky Enterprise Software Foundation, io.buildah.version=1.33.7, OSD_FLAVOR=default, license=BSD-3-Clause, version=10, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, org.opencontainers.image.licenses=BSD-3-Clause, name=rockylinux, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, summary=Rocky Linux Base image, ceph=True, CEPH_REF=wip-pdonnell-testing-20260210.171210, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.version=10) 2026-02-13T02:27:47.320 INFO:journalctl@ceph.osd.1.trial028.stdout:Feb 13 02:27:47 trial028 bash[57584]: ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-1 2026-02-13T02:27:47.320 INFO:journalctl@ceph.osd.1.trial028.stdout:Feb 13 02:27:47 trial028 podman[57718]: 2026-02-13 02:27:47.171347258 +0000 UTC m=+0.016653892 container create 17603e7097663f3d2343b05e8aa24fda0c8eacc438c8c4abdac15f52ff04af26 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-1-deactivate, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, license=BSD-3-Clause, OSD_FLAVOR=default, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, version=10, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, org.opencontainers.image.licenses=BSD-3-Clause, summary=Rocky Linux Base image, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, name=rockylinux, org.opencontainers.image.version=10, CEPH_REF=wip-pdonnell-testing-20260210.171210, ceph=True, org.opencontainers.image.title=rockylinux, io.buildah.version=1.33.7, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.documentation=https://docs.ceph.com/, vendor=Rocky Enterprise Software Foundation, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git) 2026-02-13T02:27:47.320 INFO:journalctl@ceph.osd.1.trial028.stdout:Feb 13 02:27:47 trial028 podman[57718]: 2026-02-13 02:27:47.198619966 +0000 UTC m=+0.043926580 container init 17603e7097663f3d2343b05e8aa24fda0c8eacc438c8c4abdac15f52ff04af26 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-1-deactivate, io.buildah.version=1.33.7, org.opencontainers.image.licenses=BSD-3-Clause, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, license=BSD-3-Clause, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, name=rockylinux, org.opencontainers.image.authors=Ceph Release Team , version=10, org.opencontainers.image.documentation=https://docs.ceph.com/, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, OSD_FLAVOR=default, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, vendor=Rocky Enterprise Software Foundation, summary=Rocky Linux Base image, org.opencontainers.image.version=10, CEPH_REF=wip-pdonnell-testing-20260210.171210, ceph=True, org.opencontainers.image.title=rockylinux) 2026-02-13T02:27:47.320 INFO:journalctl@ceph.osd.1.trial028.stdout:Feb 13 02:27:47 trial028 podman[57718]: 2026-02-13 02:27:47.201382846 +0000 UTC m=+0.046689480 container start 17603e7097663f3d2343b05e8aa24fda0c8eacc438c8c4abdac15f52ff04af26 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-1-deactivate, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, OSD_FLAVOR=default, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.licenses=BSD-3-Clause, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, io.buildah.version=1.33.7, version=10, ceph=True, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, name=rockylinux, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, vendor=Rocky Enterprise Software Foundation, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, license=BSD-3-Clause, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.version=10, org.opencontainers.image.title=rockylinux, CEPH_REF=wip-pdonnell-testing-20260210.171210, summary=Rocky Linux Base image) 2026-02-13T02:27:47.320 INFO:journalctl@ceph.osd.1.trial028.stdout:Feb 13 02:27:47 trial028 podman[57718]: 2026-02-13 02:27:47.201881732 +0000 UTC m=+0.047188366 container attach 17603e7097663f3d2343b05e8aa24fda0c8eacc438c8c4abdac15f52ff04af26 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-1-deactivate, vendor=Rocky Enterprise Software Foundation, summary=Rocky Linux Base image, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, io.buildah.version=1.33.7, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, OSD_FLAVOR=default, org.opencontainers.image.version=10, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.licenses=BSD-3-Clause, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, ceph=True, license=BSD-3-Clause, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.title=rockylinux, version=10, CEPH_REF=wip-pdonnell-testing-20260210.171210, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, name=rockylinux) 2026-02-13T02:27:47.320 INFO:journalctl@ceph.osd.1.trial028.stdout:Feb 13 02:27:47 trial028 podman[57718]: 2026-02-13 02:27:47.164919594 +0000 UTC m=+0.010226228 image pull b86a4a15c2dcd78db08124ee64cb48fcd681d6db032e313da19d063813b235da quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9 2026-02-13T02:27:47.348 DEBUG:teuthology.orchestra.run.trial028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.1.service' 2026-02-13T02:27:47.385 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T02:27:47.385 INFO:tasks.cephadm.osd.1:Stopped osd.1 2026-02-13T02:27:47.386 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2026-02-13T02:27:47.386 DEBUG:teuthology.orchestra.run.trial107:> sudo systemctl stop ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.2 2026-02-13T02:27:47.751 INFO:journalctl@ceph.osd.2.trial107.stdout:Feb 13 02:27:47 trial107 systemd[1]: Stopping Ceph osd.2 for 707dbb29-0882-11f1-9262-d404e6e7d460... 2026-02-13T02:27:47.751 INFO:journalctl@ceph.osd.2.trial107.stdout:Feb 13 02:27:47 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-2[32542]: 2026-02-13T02:27:47.486+0000 7fb4eb37e6c0 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --osd-objectstore=bluestore (PID: 1) UID: 0 2026-02-13T02:27:47.751 INFO:journalctl@ceph.osd.2.trial107.stdout:Feb 13 02:27:47 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-2[32542]: 2026-02-13T02:27:47.486+0000 7fb4eb37e6c0 -1 osd.2 72 *** Got signal Terminated *** 2026-02-13T02:27:47.751 INFO:journalctl@ceph.osd.2.trial107.stdout:Feb 13 02:27:47 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-2[32542]: 2026-02-13T02:27:47.486+0000 7fb4eb37e6c0 -1 osd.2 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-13T02:27:52.835 INFO:journalctl@ceph.osd.2.trial107.stdout:Feb 13 02:27:52 trial107 podman[60169]: 2026-02-13 02:27:52.545647102 +0000 UTC m=+5.071541901 container died 230bafc6aee763ed3685b9379eb96acfc44cfa06948c7468e1fa8ebed605bc16 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-2, vendor=Rocky Enterprise Software Foundation, version=10, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, io.buildah.version=1.33.7, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.title=rockylinux, org.opencontainers.image.version=10, summary=Rocky Linux Base image, CEPH_REF=wip-pdonnell-testing-20260210.171210, license=BSD-3-Clause, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, OSD_FLAVOR=default, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, ceph=True, name=rockylinux, org.opencontainers.image.licenses=BSD-3-Clause, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10) 2026-02-13T02:27:52.835 INFO:journalctl@ceph.osd.2.trial107.stdout:Feb 13 02:27:52 trial107 podman[60169]: 2026-02-13 02:27:52.554481761 +0000 UTC m=+5.080376560 container remove 230bafc6aee763ed3685b9379eb96acfc44cfa06948c7468e1fa8ebed605bc16 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-2, org.opencontainers.image.licenses=BSD-3-Clause, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, summary=Rocky Linux Base image, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, io.buildah.version=1.33.7, vendor=Rocky Enterprise Software Foundation, license=BSD-3-Clause, org.opencontainers.image.title=rockylinux, name=rockylinux, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.version=10, version=10, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, ceph=True, CEPH_REF=wip-pdonnell-testing-20260210.171210, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, OSD_FLAVOR=default, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation) 2026-02-13T02:27:52.835 INFO:journalctl@ceph.osd.2.trial107.stdout:Feb 13 02:27:52 trial107 bash[60169]: ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-2 2026-02-13T02:27:52.835 INFO:journalctl@ceph.osd.2.trial107.stdout:Feb 13 02:27:52 trial107 podman[60301]: 2026-02-13 02:27:52.684686965 +0000 UTC m=+0.016926810 container create 0c0abe38cc6d5f32c96294b3ad13380fbdd71c3082dae8611743ecae4aa23464 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-2-deactivate, license=BSD-3-Clause, OSD_FLAVOR=default, ceph=True, org.opencontainers.image.documentation=https://docs.ceph.com/, summary=Rocky Linux Base image, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, name=rockylinux, version=10, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, CEPH_REF=wip-pdonnell-testing-20260210.171210, org.opencontainers.image.title=rockylinux, vendor=Rocky Enterprise Software Foundation, io.buildah.version=1.33.7, org.opencontainers.image.authors=Ceph Release Team , CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.licenses=BSD-3-Clause, org.opencontainers.image.version=10) 2026-02-13T02:27:52.835 INFO:journalctl@ceph.osd.2.trial107.stdout:Feb 13 02:27:52 trial107 podman[60301]: 2026-02-13 02:27:52.71408106 +0000 UTC m=+0.046320835 container init 0c0abe38cc6d5f32c96294b3ad13380fbdd71c3082dae8611743ecae4aa23464 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-2-deactivate, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.licenses=BSD-3-Clause, org.opencontainers.image.version=10, ceph=True, org.opencontainers.image.documentation=https://docs.ceph.com/, version=10, license=BSD-3-Clause, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, OSD_FLAVOR=default, vendor=Rocky Enterprise Software Foundation, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, name=rockylinux, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, CEPH_REF=wip-pdonnell-testing-20260210.171210, summary=Rocky Linux Base image, org.opencontainers.image.title=rockylinux, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, io.buildah.version=1.33.7) 2026-02-13T02:27:52.835 INFO:journalctl@ceph.osd.2.trial107.stdout:Feb 13 02:27:52 trial107 podman[60301]: 2026-02-13 02:27:52.716933708 +0000 UTC m=+0.049173493 container start 0c0abe38cc6d5f32c96294b3ad13380fbdd71c3082dae8611743ecae4aa23464 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-2-deactivate, org.opencontainers.image.version=10, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, license=BSD-3-Clause, org.opencontainers.image.authors=Ceph Release Team , ceph=True, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, OSD_FLAVOR=default, vendor=Rocky Enterprise Software Foundation, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, name=rockylinux, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, version=10, org.opencontainers.image.licenses=BSD-3-Clause, io.buildah.version=1.33.7, org.opencontainers.image.documentation=https://docs.ceph.com/, summary=Rocky Linux Base image, org.opencontainers.image.title=rockylinux, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, CEPH_REF=wip-pdonnell-testing-20260210.171210) 2026-02-13T02:27:52.836 INFO:journalctl@ceph.osd.2.trial107.stdout:Feb 13 02:27:52 trial107 podman[60301]: 2026-02-13 02:27:52.717474141 +0000 UTC m=+0.049713926 container attach 0c0abe38cc6d5f32c96294b3ad13380fbdd71c3082dae8611743ecae4aa23464 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-2-deactivate, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, ceph=True, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, io.buildah.version=1.33.7, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.licenses=BSD-3-Clause, org.opencontainers.image.version=10, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, version=10, license=BSD-3-Clause, CEPH_REF=wip-pdonnell-testing-20260210.171210, summary=Rocky Linux Base image, org.opencontainers.image.title=rockylinux, name=rockylinux, OSD_FLAVOR=default, org.opencontainers.image.authors=Ceph Release Team , vendor=Rocky Enterprise Software Foundation) 2026-02-13T02:27:52.836 INFO:journalctl@ceph.osd.2.trial107.stdout:Feb 13 02:27:52 trial107 podman[60301]: 2026-02-13 02:27:52.677713363 +0000 UTC m=+0.009953148 image pull b86a4a15c2dcd78db08124ee64cb48fcd681d6db032e313da19d063813b235da quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9 2026-02-13T02:27:52.836 INFO:journalctl@ceph.osd.2.trial107.stdout:Feb 13 02:27:52 trial107 podman[60301]: 2026-02-13 02:27:52.837187218 +0000 UTC m=+0.169426993 container died 0c0abe38cc6d5f32c96294b3ad13380fbdd71c3082dae8611743ecae4aa23464 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-2-deactivate, org.opencontainers.image.version=10, vendor=Rocky Enterprise Software Foundation, io.buildah.version=1.33.7, version=10, license=BSD-3-Clause, CEPH_REF=wip-pdonnell-testing-20260210.171210, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.authors=Ceph Release Team , CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, org.opencontainers.image.licenses=BSD-3-Clause, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, OSD_FLAVOR=default, ceph=True, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.documentation=https://docs.ceph.com/, summary=Rocky Linux Base image, org.opencontainers.image.title=rockylinux, name=rockylinux) 2026-02-13T02:27:52.865 DEBUG:teuthology.orchestra.run.trial107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.2.service' 2026-02-13T02:27:52.902 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T02:27:52.902 INFO:tasks.cephadm.osd.2:Stopped osd.2 2026-02-13T02:27:52.902 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2026-02-13T02:27:52.902 DEBUG:teuthology.orchestra.run.trial027:> sudo systemctl stop ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.3 2026-02-13T02:27:53.408 INFO:journalctl@ceph.osd.3.trial027.stdout:Feb 13 02:27:52 trial027 systemd[1]: Stopping Ceph osd.3 for 707dbb29-0882-11f1-9262-d404e6e7d460... 2026-02-13T02:27:53.409 INFO:journalctl@ceph.osd.3.trial027.stdout:Feb 13 02:27:53 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-3[43064]: 2026-02-13T02:27:53.004+0000 7fcb816e56c0 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --osd-objectstore=bluestore (PID: 1) UID: 0 2026-02-13T02:27:53.409 INFO:journalctl@ceph.osd.3.trial027.stdout:Feb 13 02:27:53 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-3[43064]: 2026-02-13T02:27:53.004+0000 7fcb816e56c0 -1 osd.3 72 *** Got signal Terminated *** 2026-02-13T02:27:53.409 INFO:journalctl@ceph.osd.3.trial027.stdout:Feb 13 02:27:53 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-3[43064]: 2026-02-13T02:27:53.004+0000 7fcb816e56c0 -1 osd.3 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-13T02:27:58.365 INFO:journalctl@ceph.osd.3.trial027.stdout:Feb 13 02:27:58 trial027 podman[81614]: 2026-02-13 02:27:58.063911606 +0000 UTC m=+5.071034045 container died a0d900c4f05fe79c4b297eec0ee7dd83b856394586e7fdeac8d46c7d10dbb014 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-3, org.opencontainers.image.version=10, ceph=True, org.opencontainers.image.title=rockylinux, summary=Rocky Linux Base image, version=10, name=rockylinux, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, CEPH_REF=wip-pdonnell-testing-20260210.171210, vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.licenses=BSD-3-Clause, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, license=BSD-3-Clause, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, OSD_FLAVOR=default, io.buildah.version=1.33.7) 2026-02-13T02:27:58.366 INFO:journalctl@ceph.osd.3.trial027.stdout:Feb 13 02:27:58 trial027 podman[81614]: 2026-02-13 02:27:58.073216841 +0000 UTC m=+5.080339280 container remove a0d900c4f05fe79c4b297eec0ee7dd83b856394586e7fdeac8d46c7d10dbb014 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-3, org.opencontainers.image.authors=Ceph Release Team , name=rockylinux, org.opencontainers.image.licenses=BSD-3-Clause, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_REF=wip-pdonnell-testing-20260210.171210, ceph=True, org.opencontainers.image.title=rockylinux, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, io.buildah.version=1.33.7, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, OSD_FLAVOR=default, org.opencontainers.image.version=10, vendor=Rocky Enterprise Software Foundation, license=BSD-3-Clause, summary=Rocky Linux Base image, version=10) 2026-02-13T02:27:58.366 INFO:journalctl@ceph.osd.3.trial027.stdout:Feb 13 02:27:58 trial027 bash[81614]: ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-3 2026-02-13T02:27:58.366 INFO:journalctl@ceph.osd.3.trial027.stdout:Feb 13 02:27:58 trial027 podman[81747]: 2026-02-13 02:27:58.197605405 +0000 UTC m=+0.016260038 container create 865cc6763ddcb288cc838d4083caedc462e1165fbed8931a3ba25e57cd28492a (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-3-deactivate, org.opencontainers.image.documentation=https://docs.ceph.com/, io.buildah.version=1.33.7, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, name=rockylinux, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, license=BSD-3-Clause, org.opencontainers.image.licenses=BSD-3-Clause, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.authors=Ceph Release Team , ceph=True, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, CEPH_REF=wip-pdonnell-testing-20260210.171210, OSD_FLAVOR=default, org.opencontainers.image.version=10, vendor=Rocky Enterprise Software Foundation, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, version=10, org.opencontainers.image.title=rockylinux, summary=Rocky Linux Base image) 2026-02-13T02:27:58.366 INFO:journalctl@ceph.osd.3.trial027.stdout:Feb 13 02:27:58 trial027 podman[81747]: 2026-02-13 02:27:58.230744807 +0000 UTC m=+0.049399440 container init 865cc6763ddcb288cc838d4083caedc462e1165fbed8931a3ba25e57cd28492a (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-3-deactivate, org.opencontainers.image.authors=Ceph Release Team , CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, license=BSD-3-Clause, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, version=10, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, ceph=True, org.opencontainers.image.title=rockylinux, org.opencontainers.image.documentation=https://docs.ceph.com/, OSD_FLAVOR=default, org.opencontainers.image.version=10, io.buildah.version=1.33.7, summary=Rocky Linux Base image, name=rockylinux, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, org.opencontainers.image.licenses=BSD-3-Clause, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, CEPH_REF=wip-pdonnell-testing-20260210.171210, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, vendor=Rocky Enterprise Software Foundation) 2026-02-13T02:27:58.366 INFO:journalctl@ceph.osd.3.trial027.stdout:Feb 13 02:27:58 trial027 podman[81747]: 2026-02-13 02:27:58.233164149 +0000 UTC m=+0.051818772 container start 865cc6763ddcb288cc838d4083caedc462e1165fbed8931a3ba25e57cd28492a (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-3-deactivate, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, org.opencontainers.image.documentation=https://docs.ceph.com/, summary=Rocky Linux Base image, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, org.opencontainers.image.licenses=BSD-3-Clause, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, io.buildah.version=1.33.7, version=10, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_REF=wip-pdonnell-testing-20260210.171210, OSD_FLAVOR=default, vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.authors=Ceph Release Team , CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, name=rockylinux, ceph=True, org.opencontainers.image.version=10, license=BSD-3-Clause, org.opencontainers.image.title=rockylinux) 2026-02-13T02:27:58.366 INFO:journalctl@ceph.osd.3.trial027.stdout:Feb 13 02:27:58 trial027 podman[81747]: 2026-02-13 02:27:58.2337789 +0000 UTC m=+0.052433533 container attach 865cc6763ddcb288cc838d4083caedc462e1165fbed8931a3ba25e57cd28492a (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-3-deactivate, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, io.buildah.version=1.33.7, version=10, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, CEPH_REF=wip-pdonnell-testing-20260210.171210, OSD_FLAVOR=default, org.opencontainers.image.version=10, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, license=BSD-3-Clause, org.opencontainers.image.licenses=BSD-3-Clause, ceph=True, org.opencontainers.image.documentation=https://docs.ceph.com/, summary=Rocky Linux Base image, vendor=Rocky Enterprise Software Foundation, name=rockylinux, org.opencontainers.image.title=rockylinux, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git) 2026-02-13T02:27:58.366 INFO:journalctl@ceph.osd.3.trial027.stdout:Feb 13 02:27:58 trial027 podman[81747]: 2026-02-13 02:27:58.191362946 +0000 UTC m=+0.010017579 image pull b86a4a15c2dcd78db08124ee64cb48fcd681d6db032e313da19d063813b235da quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9 2026-02-13T02:27:58.396 DEBUG:teuthology.orchestra.run.trial027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.3.service' 2026-02-13T02:27:58.433 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T02:27:58.433 INFO:tasks.cephadm.osd.3:Stopped osd.3 2026-02-13T02:27:58.434 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2026-02-13T02:27:58.434 DEBUG:teuthology.orchestra.run.trial028:> sudo systemctl stop ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.4 2026-02-13T02:27:58.743 INFO:journalctl@ceph.osd.4.trial028.stdout:Feb 13 02:27:58 trial028 systemd[1]: Stopping Ceph osd.4 for 707dbb29-0882-11f1-9262-d404e6e7d460... 2026-02-13T02:27:58.743 INFO:journalctl@ceph.osd.4.trial028.stdout:Feb 13 02:27:58 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-4[37681]: 2026-02-13T02:27:58.535+0000 7f37e5e296c0 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --osd-objectstore=bluestore (PID: 1) UID: 0 2026-02-13T02:27:58.743 INFO:journalctl@ceph.osd.4.trial028.stdout:Feb 13 02:27:58 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-4[37681]: 2026-02-13T02:27:58.535+0000 7f37e5e296c0 -1 osd.4 72 *** Got signal Terminated *** 2026-02-13T02:27:58.743 INFO:journalctl@ceph.osd.4.trial028.stdout:Feb 13 02:27:58 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-4[37681]: 2026-02-13T02:27:58.535+0000 7f37e5e296c0 -1 osd.4 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-13T02:28:03.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:02 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:02.589+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:03.158 INFO:journalctl@ceph.osd.6.trial027.stdout:Feb 13 02:28:02 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-6[49552]: 2026-02-13T02:28:02.790+0000 7f710044f6c0 -1 osd.6 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.680978+0000 front 2026-02-13T02:27:37.680986+0000 (oldest deadline 2026-02-13T02:28:02.381056+0000) 2026-02-13T02:28:03.889 INFO:journalctl@ceph.osd.4.trial028.stdout:Feb 13 02:28:03 trial028 podman[57857]: 2026-02-13 02:28:03.59576306 +0000 UTC m=+5.071492337 container died 027e66c59d2be8f761c9724349f8a521bce19d27e76a02762df390c68dcc0977 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-4, CEPH_REF=wip-pdonnell-testing-20260210.171210, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.title=rockylinux, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, OSD_FLAVOR=default, ceph=True, license=BSD-3-Clause, name=rockylinux, org.opencontainers.image.licenses=BSD-3-Clause, org.opencontainers.image.version=10, version=10, vendor=Rocky Enterprise Software Foundation, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, io.buildah.version=1.33.7, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, summary=Rocky Linux Base image) 2026-02-13T02:28:03.890 INFO:journalctl@ceph.osd.4.trial028.stdout:Feb 13 02:28:03 trial028 podman[57857]: 2026-02-13 02:28:03.605047439 +0000 UTC m=+5.080776706 container remove 027e66c59d2be8f761c9724349f8a521bce19d27e76a02762df390c68dcc0977 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-4, CEPH_REF=wip-pdonnell-testing-20260210.171210, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.licenses=BSD-3-Clause, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.documentation=https://docs.ceph.com/, summary=Rocky Linux Base image, org.opencontainers.image.title=rockylinux, license=BSD-3-Clause, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, vendor=Rocky Enterprise Software Foundation, io.buildah.version=1.33.7, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.version=10, name=rockylinux, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, version=10, OSD_FLAVOR=default, ceph=True) 2026-02-13T02:28:03.890 INFO:journalctl@ceph.osd.4.trial028.stdout:Feb 13 02:28:03 trial028 bash[57857]: ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-4 2026-02-13T02:28:03.890 INFO:journalctl@ceph.osd.4.trial028.stdout:Feb 13 02:28:03 trial028 podman[57989]: 2026-02-13 02:28:03.737005728 +0000 UTC m=+0.015046398 container create 5d520c048563da4d30405df4f0eda33a228177c10cbd6399b3ea8da9017ec30e (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-4-deactivate, name=rockylinux, org.opencontainers.image.authors=Ceph Release Team , CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.title=rockylinux, org.opencontainers.image.licenses=BSD-3-Clause, CEPH_REF=wip-pdonnell-testing-20260210.171210, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, version=10, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, ceph=True, license=BSD-3-Clause, org.opencontainers.image.version=10, io.buildah.version=1.33.7, OSD_FLAVOR=default, org.opencontainers.image.documentation=https://docs.ceph.com/, vendor=Rocky Enterprise Software Foundation, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, summary=Rocky Linux Base image) 2026-02-13T02:28:03.890 INFO:journalctl@ceph.osd.4.trial028.stdout:Feb 13 02:28:03 trial028 podman[57989]: 2026-02-13 02:28:03.76949463 +0000 UTC m=+0.047535240 container init 5d520c048563da4d30405df4f0eda33a228177c10cbd6399b3ea8da9017ec30e (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-4-deactivate, org.opencontainers.image.documentation=https://docs.ceph.com/, vendor=Rocky Enterprise Software Foundation, OSD_FLAVOR=default, ceph=True, name=rockylinux, summary=Rocky Linux Base image, io.buildah.version=1.33.7, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, CEPH_REF=wip-pdonnell-testing-20260210.171210, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, license=BSD-3-Clause, org.opencontainers.image.version=10, version=10, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.title=rockylinux, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.licenses=BSD-3-Clause) 2026-02-13T02:28:03.890 INFO:journalctl@ceph.osd.4.trial028.stdout:Feb 13 02:28:03 trial028 podman[57989]: 2026-02-13 02:28:03.772281004 +0000 UTC m=+0.050321634 container start 5d520c048563da4d30405df4f0eda33a228177c10cbd6399b3ea8da9017ec30e (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-4-deactivate, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.title=rockylinux, org.opencontainers.image.licenses=BSD-3-Clause, name=rockylinux, license=BSD-3-Clause, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, CEPH_REF=wip-pdonnell-testing-20260210.171210, org.opencontainers.image.documentation=https://docs.ceph.com/, ceph=True, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, version=10, summary=Rocky Linux Base image, OSD_FLAVOR=default, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, io.buildah.version=1.33.7, vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.version=10) 2026-02-13T02:28:03.890 INFO:journalctl@ceph.osd.4.trial028.stdout:Feb 13 02:28:03 trial028 podman[57989]: 2026-02-13 02:28:03.772824703 +0000 UTC m=+0.050865323 container attach 5d520c048563da4d30405df4f0eda33a228177c10cbd6399b3ea8da9017ec30e (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-4-deactivate, org.opencontainers.image.authors=Ceph Release Team , OSD_FLAVOR=default, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, name=rockylinux, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, org.opencontainers.image.version=10, io.buildah.version=1.33.7, org.opencontainers.image.title=rockylinux, org.opencontainers.image.licenses=BSD-3-Clause, org.opencontainers.image.documentation=https://docs.ceph.com/, ceph=True, version=10, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_REF=wip-pdonnell-testing-20260210.171210, vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, license=BSD-3-Clause, summary=Rocky Linux Base image) 2026-02-13T02:28:03.890 INFO:journalctl@ceph.osd.4.trial028.stdout:Feb 13 02:28:03 trial028 podman[57989]: 2026-02-13 02:28:03.730958289 +0000 UTC m=+0.008998899 image pull b86a4a15c2dcd78db08124ee64cb48fcd681d6db032e313da19d063813b235da quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9 2026-02-13T02:28:03.917 DEBUG:teuthology.orchestra.run.trial028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.4.service' 2026-02-13T02:28:03.954 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T02:28:03.954 INFO:tasks.cephadm.osd.4:Stopped osd.4 2026-02-13T02:28:03.954 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2026-02-13T02:28:03.954 DEBUG:teuthology.orchestra.run.trial107:> sudo systemctl stop ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.5 2026-02-13T02:28:03.978 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:03 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:03.559+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:04.059 INFO:journalctl@ceph.osd.6.trial027.stdout:Feb 13 02:28:03 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-6[49552]: 2026-02-13T02:28:03.749+0000 7f710044f6c0 -1 osd.6 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.680978+0000 front 2026-02-13T02:27:37.680986+0000 (oldest deadline 2026-02-13T02:28:02.381056+0000) 2026-02-13T02:28:04.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:04 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:04.051+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:04.251 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:04 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:04.162+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.778102+0000 front 2026-02-13T02:27:37.778172+0000 (oldest deadline 2026-02-13T02:28:03.678195+0000) 2026-02-13T02:28:04.251 INFO:journalctl@ceph.osd.5.trial107.stdout:Feb 13 02:28:03 trial107 systemd[1]: Stopping Ceph osd.5 for 707dbb29-0882-11f1-9262-d404e6e7d460... 2026-02-13T02:28:04.251 INFO:journalctl@ceph.osd.5.trial107.stdout:Feb 13 02:28:04 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-5[38600]: 2026-02-13T02:28:04.055+0000 7ff210c8e6c0 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --osd-objectstore=bluestore (PID: 1) UID: 0 2026-02-13T02:28:04.251 INFO:journalctl@ceph.osd.5.trial107.stdout:Feb 13 02:28:04 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-5[38600]: 2026-02-13T02:28:04.055+0000 7ff210c8e6c0 -1 osd.5 72 *** Got signal Terminated *** 2026-02-13T02:28:04.251 INFO:journalctl@ceph.osd.5.trial107.stdout:Feb 13 02:28:04 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-5[38600]: 2026-02-13T02:28:04.055+0000 7ff210c8e6c0 -1 osd.5 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-13T02:28:04.908 INFO:journalctl@ceph.osd.6.trial027.stdout:Feb 13 02:28:04 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-6[49552]: 2026-02-13T02:28:04.753+0000 7f710044f6c0 -1 osd.6 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.680978+0000 front 2026-02-13T02:27:37.680986+0000 (oldest deadline 2026-02-13T02:28:02.381056+0000) 2026-02-13T02:28:04.909 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:04 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:04.571+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:39.585067+0000 front 2026-02-13T02:27:39.585049+0000 (oldest deadline 2026-02-13T02:28:03.685105+0000) 2026-02-13T02:28:05.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:04 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:04.545+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:05.493 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:05 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:05.087+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:05.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:05 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:05.209+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.778102+0000 front 2026-02-13T02:27:37.778172+0000 (oldest deadline 2026-02-13T02:28:03.678195+0000) 2026-02-13T02:28:05.501 INFO:journalctl@ceph.osd.5.trial107.stdout:Feb 13 02:28:05 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-5[38600]: 2026-02-13T02:28:05.079+0000 7ff20d3536c0 -1 osd.5 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.332955+0000 front 2026-02-13T02:27:40.332976+0000 (oldest deadline 2026-02-13T02:28:04.433049+0000) 2026-02-13T02:28:05.908 INFO:journalctl@ceph.osd.6.trial027.stdout:Feb 13 02:28:05 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-6[49552]: 2026-02-13T02:28:05.798+0000 7f710044f6c0 -1 osd.6 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.680978+0000 front 2026-02-13T02:27:37.680986+0000 (oldest deadline 2026-02-13T02:28:02.381056+0000) 2026-02-13T02:28:05.908 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:05 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:05.615+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:39.585067+0000 front 2026-02-13T02:27:39.585049+0000 (oldest deadline 2026-02-13T02:28:03.685105+0000) 2026-02-13T02:28:06.000 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:05 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:05.563+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:06.493 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:06 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:06.086+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:06.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:06 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:06.197+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.778102+0000 front 2026-02-13T02:27:37.778172+0000 (oldest deadline 2026-02-13T02:28:03.678195+0000) 2026-02-13T02:28:06.501 INFO:journalctl@ceph.osd.5.trial107.stdout:Feb 13 02:28:06 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-5[38600]: 2026-02-13T02:28:06.054+0000 7ff20d3536c0 -1 osd.5 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.332955+0000 front 2026-02-13T02:27:40.332976+0000 (oldest deadline 2026-02-13T02:28:04.433049+0000) 2026-02-13T02:28:06.908 INFO:journalctl@ceph.osd.6.trial027.stdout:Feb 13 02:28:06 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-6[49552]: 2026-02-13T02:28:06.757+0000 7f710044f6c0 -1 osd.6 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.680978+0000 front 2026-02-13T02:27:37.680986+0000 (oldest deadline 2026-02-13T02:28:02.381056+0000) 2026-02-13T02:28:06.909 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:06 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:06.633+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:39.585067+0000 front 2026-02-13T02:27:39.585049+0000 (oldest deadline 2026-02-13T02:28:03.685105+0000) 2026-02-13T02:28:07.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:06 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:06.613+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:07.493 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:07 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:07.092+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:07.501 INFO:journalctl@ceph.osd.5.trial107.stdout:Feb 13 02:28:07 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-5[38600]: 2026-02-13T02:28:07.042+0000 7ff20d3536c0 -1 osd.5 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.332955+0000 front 2026-02-13T02:27:40.332976+0000 (oldest deadline 2026-02-13T02:28:04.433049+0000) 2026-02-13T02:28:07.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:07 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:07.182+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.778102+0000 front 2026-02-13T02:27:37.778172+0000 (oldest deadline 2026-02-13T02:28:03.678195+0000) 2026-02-13T02:28:08.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:07 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:07.627+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:08.158 INFO:journalctl@ceph.osd.6.trial027.stdout:Feb 13 02:28:07 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-6[49552]: 2026-02-13T02:28:07.718+0000 7f710044f6c0 -1 osd.6 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.680978+0000 front 2026-02-13T02:27:37.680986+0000 (oldest deadline 2026-02-13T02:28:02.381056+0000) 2026-02-13T02:28:08.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:07 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:07.678+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:39.585067+0000 front 2026-02-13T02:27:39.585049+0000 (oldest deadline 2026-02-13T02:28:03.685105+0000) 2026-02-13T02:28:08.493 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:08 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:08.055+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:08.493 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:08 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:08.055+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.710229+0000 front 2026-02-13T02:27:43.710211+0000 (oldest deadline 2026-02-13T02:28:07.210302+0000) 2026-02-13T02:28:08.501 INFO:journalctl@ceph.osd.5.trial107.stdout:Feb 13 02:28:08 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-5[38600]: 2026-02-13T02:28:08.059+0000 7ff20d3536c0 -1 osd.5 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.332955+0000 front 2026-02-13T02:27:40.332976+0000 (oldest deadline 2026-02-13T02:28:04.433049+0000) 2026-02-13T02:28:08.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:08 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:08.191+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.778102+0000 front 2026-02-13T02:27:37.778172+0000 (oldest deadline 2026-02-13T02:28:03.678195+0000) 2026-02-13T02:28:08.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:08 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:08.191+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:44.778785+0000 front 2026-02-13T02:27:44.778712+0000 (oldest deadline 2026-02-13T02:28:07.678732+0000) 2026-02-13T02:28:09.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:08 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:08.658+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:09.158 INFO:journalctl@ceph.osd.6.trial027.stdout:Feb 13 02:28:08 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-6[49552]: 2026-02-13T02:28:08.698+0000 7f710044f6c0 -1 osd.6 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.680978+0000 front 2026-02-13T02:27:37.680986+0000 (oldest deadline 2026-02-13T02:28:02.381056+0000) 2026-02-13T02:28:09.159 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:08 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:08.706+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:39.585067+0000 front 2026-02-13T02:27:39.585049+0000 (oldest deadline 2026-02-13T02:28:03.685105+0000) 2026-02-13T02:28:09.159 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:08 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:08.706+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.685350+0000 front 2026-02-13T02:27:43.685369+0000 (oldest deadline 2026-02-13T02:28:08.385425+0000) 2026-02-13T02:28:09.415 INFO:journalctl@ceph.osd.5.trial107.stdout:Feb 13 02:28:09 trial107 podman[60435]: 2026-02-13 02:28:09.125319549 +0000 UTC m=+5.082203521 container died 19123bf46d163144385fc6f7d43b545cce5988c44d0747acdad60f9fed37e5fc (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-5, org.opencontainers.image.title=rockylinux, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, org.opencontainers.image.version=10, name=rockylinux, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, io.buildah.version=1.33.7, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, org.opencontainers.image.documentation=https://docs.ceph.com/, version=10, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, vendor=Rocky Enterprise Software Foundation, OSD_FLAVOR=default, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, summary=Rocky Linux Base image, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.licenses=BSD-3-Clause, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, license=BSD-3-Clause, CEPH_REF=wip-pdonnell-testing-20260210.171210, ceph=True) 2026-02-13T02:28:09.415 INFO:journalctl@ceph.osd.5.trial107.stdout:Feb 13 02:28:09 trial107 podman[60435]: 2026-02-13 02:28:09.133907259 +0000 UTC m=+5.090791221 container remove 19123bf46d163144385fc6f7d43b545cce5988c44d0747acdad60f9fed37e5fc (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-5, org.opencontainers.image.authors=Ceph Release Team , vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, CEPH_REF=wip-pdonnell-testing-20260210.171210, name=rockylinux, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, license=BSD-3-Clause, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, org.opencontainers.image.licenses=BSD-3-Clause, org.opencontainers.image.version=10, version=10, OSD_FLAVOR=default, io.buildah.version=1.33.7, summary=Rocky Linux Base image, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, ceph=True, org.opencontainers.image.title=rockylinux) 2026-02-13T02:28:09.415 INFO:journalctl@ceph.osd.5.trial107.stdout:Feb 13 02:28:09 trial107 bash[60435]: ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-5 2026-02-13T02:28:09.415 INFO:journalctl@ceph.osd.5.trial107.stdout:Feb 13 02:28:09 trial107 podman[60566]: 2026-02-13 02:28:09.264735714 +0000 UTC m=+0.015873332 container create 08e7ec2df05765cc28356a630dc786d02c59910dd89760083ae242a5c000175a (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-5-deactivate, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, OSD_FLAVOR=default, summary=Rocky Linux Base image, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, org.opencontainers.image.version=10, version=10, io.buildah.version=1.33.7, name=rockylinux, ceph=True, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.licenses=BSD-3-Clause, license=BSD-3-Clause, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.title=rockylinux, CEPH_REF=wip-pdonnell-testing-20260210.171210) 2026-02-13T02:28:09.416 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:09 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:09.154+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.778102+0000 front 2026-02-13T02:27:37.778172+0000 (oldest deadline 2026-02-13T02:28:03.678195+0000) 2026-02-13T02:28:09.416 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:09 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:09.154+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:44.778785+0000 front 2026-02-13T02:27:44.778712+0000 (oldest deadline 2026-02-13T02:28:07.678732+0000) 2026-02-13T02:28:09.416 INFO:journalctl@ceph.osd.5.trial107.stdout:Feb 13 02:28:09 trial107 podman[60566]: 2026-02-13 02:28:09.296337066 +0000 UTC m=+0.047474714 container init 08e7ec2df05765cc28356a630dc786d02c59910dd89760083ae242a5c000175a (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-5-deactivate, name=rockylinux, io.buildah.version=1.33.7, org.opencontainers.image.version=10, version=10, org.opencontainers.image.title=rockylinux, OSD_FLAVOR=default, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, ceph=True, license=BSD-3-Clause, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_REF=wip-pdonnell-testing-20260210.171210, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.licenses=BSD-3-Clause, org.opencontainers.image.authors=Ceph Release Team , CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, vendor=Rocky Enterprise Software Foundation, summary=Rocky Linux Base image) 2026-02-13T02:28:09.416 INFO:journalctl@ceph.osd.5.trial107.stdout:Feb 13 02:28:09 trial107 podman[60566]: 2026-02-13 02:28:09.298914954 +0000 UTC m=+0.050052582 container start 08e7ec2df05765cc28356a630dc786d02c59910dd89760083ae242a5c000175a (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-5-deactivate, license=BSD-3-Clause, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_REF=wip-pdonnell-testing-20260210.171210, name=rockylinux, org.opencontainers.image.version=10, org.opencontainers.image.licenses=BSD-3-Clause, org.opencontainers.image.title=rockylinux, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, org.opencontainers.image.documentation=https://docs.ceph.com/, vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, version=10, ceph=True, io.buildah.version=1.33.7, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, OSD_FLAVOR=default, summary=Rocky Linux Base image) 2026-02-13T02:28:09.416 INFO:journalctl@ceph.osd.5.trial107.stdout:Feb 13 02:28:09 trial107 podman[60566]: 2026-02-13 02:28:09.299364499 +0000 UTC m=+0.050502127 container attach 08e7ec2df05765cc28356a630dc786d02c59910dd89760083ae242a5c000175a (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-5-deactivate, org.opencontainers.image.authors=Ceph Release Team , CEPH_REF=wip-pdonnell-testing-20260210.171210, OSD_FLAVOR=default, summary=Rocky Linux Base image, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, org.opencontainers.image.title=rockylinux, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, vendor=Rocky Enterprise Software Foundation, ceph=True, org.opencontainers.image.documentation=https://docs.ceph.com/, io.buildah.version=1.33.7, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, name=rockylinux, org.opencontainers.image.licenses=BSD-3-Clause, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.version=10, version=10, license=BSD-3-Clause) 2026-02-13T02:28:09.416 INFO:journalctl@ceph.osd.5.trial107.stdout:Feb 13 02:28:09 trial107 podman[60566]: 2026-02-13 02:28:09.258220403 +0000 UTC m=+0.009358031 image pull b86a4a15c2dcd78db08124ee64cb48fcd681d6db032e313da19d063813b235da quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9 2026-02-13T02:28:09.444 DEBUG:teuthology.orchestra.run.trial107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.5.service' 2026-02-13T02:28:09.493 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:09 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:09.019+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:09.493 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:09 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:09.019+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.710229+0000 front 2026-02-13T02:27:43.710211+0000 (oldest deadline 2026-02-13T02:28:07.210302+0000) 2026-02-13T02:28:09.520 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T02:28:09.521 INFO:tasks.cephadm.osd.5:Stopped osd.5 2026-02-13T02:28:09.521 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2026-02-13T02:28:09.521 DEBUG:teuthology.orchestra.run.trial027:> sudo systemctl stop ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.6 2026-02-13T02:28:09.751 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:09 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:09.614+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:09.908 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:09 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:09.752+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:39.585067+0000 front 2026-02-13T02:27:39.585049+0000 (oldest deadline 2026-02-13T02:28:03.685105+0000) 2026-02-13T02:28:09.908 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:09 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:09.752+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.685350+0000 front 2026-02-13T02:27:43.685369+0000 (oldest deadline 2026-02-13T02:28:08.385425+0000) 2026-02-13T02:28:09.909 INFO:journalctl@ceph.osd.6.trial027.stdout:Feb 13 02:28:09 trial027 systemd[1]: Stopping Ceph osd.6 for 707dbb29-0882-11f1-9262-d404e6e7d460... 2026-02-13T02:28:09.909 INFO:journalctl@ceph.osd.6.trial027.stdout:Feb 13 02:28:09 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-6[49552]: 2026-02-13T02:28:09.621+0000 7f7103d8a6c0 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --osd-objectstore=bluestore (PID: 1) UID: 0 2026-02-13T02:28:09.909 INFO:journalctl@ceph.osd.6.trial027.stdout:Feb 13 02:28:09 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-6[49552]: 2026-02-13T02:28:09.621+0000 7f7103d8a6c0 -1 osd.6 72 *** Got signal Terminated *** 2026-02-13T02:28:09.909 INFO:journalctl@ceph.osd.6.trial027.stdout:Feb 13 02:28:09 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-6[49552]: 2026-02-13T02:28:09.621+0000 7f7103d8a6c0 -1 osd.6 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-13T02:28:09.909 INFO:journalctl@ceph.osd.6.trial027.stdout:Feb 13 02:28:09 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-6[49552]: 2026-02-13T02:28:09.722+0000 7f710044f6c0 -1 osd.6 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.680978+0000 front 2026-02-13T02:27:37.680986+0000 (oldest deadline 2026-02-13T02:28:02.381056+0000) 2026-02-13T02:28:10.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:09 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:09.995+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:10.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:09 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:09.995+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.710229+0000 front 2026-02-13T02:27:43.710211+0000 (oldest deadline 2026-02-13T02:28:07.210302+0000) 2026-02-13T02:28:10.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:10 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:10.188+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.778102+0000 front 2026-02-13T02:27:37.778172+0000 (oldest deadline 2026-02-13T02:28:03.678195+0000) 2026-02-13T02:28:10.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:10 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:10.188+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:44.778785+0000 front 2026-02-13T02:27:44.778712+0000 (oldest deadline 2026-02-13T02:28:07.678732+0000) 2026-02-13T02:28:11.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:10 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:10.612+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:11.158 INFO:journalctl@ceph.osd.6.trial027.stdout:Feb 13 02:28:10 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-6[49552]: 2026-02-13T02:28:10.745+0000 7f710044f6c0 -1 osd.6 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.680978+0000 front 2026-02-13T02:27:37.680986+0000 (oldest deadline 2026-02-13T02:28:02.381056+0000) 2026-02-13T02:28:11.159 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:10 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:10.764+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:39.585067+0000 front 2026-02-13T02:27:39.585049+0000 (oldest deadline 2026-02-13T02:28:03.685105+0000) 2026-02-13T02:28:11.159 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:10 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:10.764+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.685350+0000 front 2026-02-13T02:27:43.685369+0000 (oldest deadline 2026-02-13T02:28:08.385425+0000) 2026-02-13T02:28:11.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:10 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:10.968+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:11.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:10 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:10.968+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.710229+0000 front 2026-02-13T02:27:43.710211+0000 (oldest deadline 2026-02-13T02:28:07.210302+0000) 2026-02-13T02:28:11.243 INFO:journalctl@ceph.osd.7.trial028.stdout:Feb 13 02:28:10 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-7[44087]: 2026-02-13T02:28:10.955+0000 7f10ff88a6c0 -1 osd.7 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:45.387784+0000 front 2026-02-13T02:27:45.387797+0000 (oldest deadline 2026-02-13T02:28:10.687895+0000) 2026-02-13T02:28:11.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:11 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:11.186+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.778102+0000 front 2026-02-13T02:27:37.778172+0000 (oldest deadline 2026-02-13T02:28:03.678195+0000) 2026-02-13T02:28:11.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:11 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:11.186+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:44.778785+0000 front 2026-02-13T02:27:44.778712+0000 (oldest deadline 2026-02-13T02:28:07.678732+0000) 2026-02-13T02:28:12.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:11 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:11.642+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:12.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:11 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:11.642+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:46.873217+0000 front 2026-02-13T02:27:46.873042+0000 (oldest deadline 2026-02-13T02:28:10.972948+0000) 2026-02-13T02:28:12.158 INFO:journalctl@ceph.osd.6.trial027.stdout:Feb 13 02:28:11 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-6[49552]: 2026-02-13T02:28:11.702+0000 7f710044f6c0 -1 osd.6 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.680978+0000 front 2026-02-13T02:27:37.680986+0000 (oldest deadline 2026-02-13T02:28:02.381056+0000) 2026-02-13T02:28:12.159 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:11 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:11.786+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:39.585067+0000 front 2026-02-13T02:27:39.585049+0000 (oldest deadline 2026-02-13T02:28:03.685105+0000) 2026-02-13T02:28:12.159 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:11 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:11.786+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.685350+0000 front 2026-02-13T02:27:43.685369+0000 (oldest deadline 2026-02-13T02:28:08.385425+0000) 2026-02-13T02:28:12.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:12 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:12.015+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:12.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:12 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:12.015+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.710229+0000 front 2026-02-13T02:27:43.710211+0000 (oldest deadline 2026-02-13T02:28:07.210302+0000) 2026-02-13T02:28:12.243 INFO:journalctl@ceph.osd.7.trial028.stdout:Feb 13 02:28:11 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-7[44087]: 2026-02-13T02:28:11.968+0000 7f10ff88a6c0 -1 osd.7 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:45.387784+0000 front 2026-02-13T02:27:45.387797+0000 (oldest deadline 2026-02-13T02:28:10.687895+0000) 2026-02-13T02:28:12.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:12 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:12.175+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.778102+0000 front 2026-02-13T02:27:37.778172+0000 (oldest deadline 2026-02-13T02:28:03.678195+0000) 2026-02-13T02:28:12.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:12 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:12.175+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:44.778785+0000 front 2026-02-13T02:27:44.778712+0000 (oldest deadline 2026-02-13T02:28:07.678732+0000) 2026-02-13T02:28:13.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:12 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:12.651+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:13.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:12 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:12.651+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:46.873217+0000 front 2026-02-13T02:27:46.873042+0000 (oldest deadline 2026-02-13T02:28:10.972948+0000) 2026-02-13T02:28:13.158 INFO:journalctl@ceph.osd.6.trial027.stdout:Feb 13 02:28:12 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-6[49552]: 2026-02-13T02:28:12.699+0000 7f710044f6c0 -1 osd.6 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.680978+0000 front 2026-02-13T02:27:37.680986+0000 (oldest deadline 2026-02-13T02:28:02.381056+0000) 2026-02-13T02:28:13.159 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:12 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:12.751+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:39.585067+0000 front 2026-02-13T02:27:39.585049+0000 (oldest deadline 2026-02-13T02:28:03.685105+0000) 2026-02-13T02:28:13.159 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:12 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:12.751+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.685350+0000 front 2026-02-13T02:27:43.685369+0000 (oldest deadline 2026-02-13T02:28:08.385425+0000) 2026-02-13T02:28:13.243 INFO:journalctl@ceph.osd.7.trial028.stdout:Feb 13 02:28:12 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-7[44087]: 2026-02-13T02:28:12.949+0000 7f10ff88a6c0 -1 osd.7 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:45.387784+0000 front 2026-02-13T02:27:45.387797+0000 (oldest deadline 2026-02-13T02:28:10.687895+0000) 2026-02-13T02:28:13.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:13 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:13.059+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:13.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:13 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:13.059+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.710229+0000 front 2026-02-13T02:27:43.710211+0000 (oldest deadline 2026-02-13T02:28:07.210302+0000) 2026-02-13T02:28:13.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:13 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:13.134+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.778102+0000 front 2026-02-13T02:27:37.778172+0000 (oldest deadline 2026-02-13T02:28:03.678195+0000) 2026-02-13T02:28:13.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:13 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:13.134+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:44.778785+0000 front 2026-02-13T02:27:44.778712+0000 (oldest deadline 2026-02-13T02:28:07.678732+0000) 2026-02-13T02:28:14.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:13 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:13.659+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:14.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:13 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:13.659+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:46.873217+0000 front 2026-02-13T02:27:46.873042+0000 (oldest deadline 2026-02-13T02:28:10.972948+0000) 2026-02-13T02:28:14.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:13 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:13.774+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:39.585067+0000 front 2026-02-13T02:27:39.585049+0000 (oldest deadline 2026-02-13T02:28:03.685105+0000) 2026-02-13T02:28:14.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:13 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:13.774+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.685350+0000 front 2026-02-13T02:27:43.685369+0000 (oldest deadline 2026-02-13T02:28:08.385425+0000) 2026-02-13T02:28:14.159 INFO:journalctl@ceph.osd.6.trial027.stdout:Feb 13 02:28:13 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-6[49552]: 2026-02-13T02:28:13.698+0000 7f710044f6c0 -1 osd.6 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.680978+0000 front 2026-02-13T02:27:37.680986+0000 (oldest deadline 2026-02-13T02:28:02.381056+0000) 2026-02-13T02:28:14.159 INFO:journalctl@ceph.osd.6.trial027.stdout:Feb 13 02:28:13 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-6[49552]: 2026-02-13T02:28:13.698+0000 7f710044f6c0 -1 osd.6 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:44.081541+0000 front 2026-02-13T02:27:44.081487+0000 (oldest deadline 2026-02-13T02:28:12.881699+0000) 2026-02-13T02:28:14.159 INFO:journalctl@ceph.osd.6.trial027.stdout:Feb 13 02:28:13 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-6[49552]: 2026-02-13T02:28:13.698+0000 7f710044f6c0 -1 osd.6 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:46.981702+0000 front 2026-02-13T02:27:46.981714+0000 (oldest deadline 2026-02-13T02:28:12.881699+0000) 2026-02-13T02:28:14.243 INFO:journalctl@ceph.osd.7.trial028.stdout:Feb 13 02:28:13 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-7[44087]: 2026-02-13T02:28:13.967+0000 7f10ff88a6c0 -1 osd.7 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:45.387784+0000 front 2026-02-13T02:27:45.387797+0000 (oldest deadline 2026-02-13T02:28:10.687895+0000) 2026-02-13T02:28:14.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:14 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:14.041+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:14.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:14 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:14.041+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.710229+0000 front 2026-02-13T02:27:43.710211+0000 (oldest deadline 2026-02-13T02:28:07.210302+0000) 2026-02-13T02:28:14.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:14 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:14.125+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.778102+0000 front 2026-02-13T02:27:37.778172+0000 (oldest deadline 2026-02-13T02:28:03.678195+0000) 2026-02-13T02:28:14.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:14 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:14.125+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:44.778785+0000 front 2026-02-13T02:27:44.778712+0000 (oldest deadline 2026-02-13T02:28:07.678732+0000) 2026-02-13T02:28:14.973 INFO:journalctl@ceph.osd.6.trial027.stdout:Feb 13 02:28:14 trial027 podman[81884]: 2026-02-13 02:28:14.681012334 +0000 UTC m=+5.072389226 container died 8ebbc9b0a04c3bb75be765696461ea9f319f3573782258c0f024bd40b609f6c5 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-6, license=BSD-3-Clause, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.licenses=BSD-3-Clause, org.opencontainers.image.version=10, CEPH_REF=wip-pdonnell-testing-20260210.171210, OSD_FLAVOR=default, io.buildah.version=1.33.7, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, vendor=Rocky Enterprise Software Foundation, version=10, org.opencontainers.image.authors=Ceph Release Team , name=rockylinux, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, org.opencontainers.image.title=rockylinux, summary=Rocky Linux Base image, ceph=True) 2026-02-13T02:28:14.973 INFO:journalctl@ceph.osd.6.trial027.stdout:Feb 13 02:28:14 trial027 podman[81884]: 2026-02-13 02:28:14.690341986 +0000 UTC m=+5.081718878 container remove 8ebbc9b0a04c3bb75be765696461ea9f319f3573782258c0f024bd40b609f6c5 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-6, vendor=Rocky Enterprise Software Foundation, io.buildah.version=1.33.7, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.title=rockylinux, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.licenses=BSD-3-Clause, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, OSD_FLAVOR=default, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, summary=Rocky Linux Base image, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, license=BSD-3-Clause, org.opencontainers.image.version=10, version=10, CEPH_REF=wip-pdonnell-testing-20260210.171210, ceph=True, name=rockylinux) 2026-02-13T02:28:14.974 INFO:journalctl@ceph.osd.6.trial027.stdout:Feb 13 02:28:14 trial027 bash[81884]: ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-6 2026-02-13T02:28:14.974 INFO:journalctl@ceph.osd.6.trial027.stdout:Feb 13 02:28:14 trial027 podman[82012]: 2026-02-13 02:28:14.81830013 +0000 UTC m=+0.016400863 container create 3eeee811732385c8771ed06d01170192de227a9a4e4cd696f379cdaeb9592022 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-6-deactivate, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.version=10, io.buildah.version=1.33.7, license=BSD-3-Clause, summary=Rocky Linux Base image, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.authors=Ceph Release Team , vendor=Rocky Enterprise Software Foundation, CEPH_REF=wip-pdonnell-testing-20260210.171210, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, OSD_FLAVOR=default, org.opencontainers.image.licenses=BSD-3-Clause, ceph=True, org.opencontainers.image.documentation=https://docs.ceph.com/, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.title=rockylinux, name=rockylinux, version=10) 2026-02-13T02:28:14.974 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:14 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:14.805+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:39.585067+0000 front 2026-02-13T02:27:39.585049+0000 (oldest deadline 2026-02-13T02:28:03.685105+0000) 2026-02-13T02:28:14.974 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:14 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:14.805+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.685350+0000 front 2026-02-13T02:27:43.685369+0000 (oldest deadline 2026-02-13T02:28:08.385425+0000) 2026-02-13T02:28:14.975 INFO:journalctl@ceph.osd.6.trial027.stdout:Feb 13 02:28:14 trial027 podman[82012]: 2026-02-13 02:28:14.846660707 +0000 UTC m=+0.044761440 container init 3eeee811732385c8771ed06d01170192de227a9a4e4cd696f379cdaeb9592022 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-6-deactivate, org.opencontainers.image.authors=Ceph Release Team , CEPH_REF=wip-pdonnell-testing-20260210.171210, org.opencontainers.image.documentation=https://docs.ceph.com/, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, org.opencontainers.image.title=rockylinux, license=BSD-3-Clause, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.version=10, OSD_FLAVOR=default, org.opencontainers.image.licenses=BSD-3-Clause, io.buildah.version=1.33.7, version=10, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, ceph=True, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, summary=Rocky Linux Base image, vendor=Rocky Enterprise Software Foundation, name=rockylinux) 2026-02-13T02:28:14.975 INFO:journalctl@ceph.osd.6.trial027.stdout:Feb 13 02:28:14 trial027 podman[82012]: 2026-02-13 02:28:14.848976728 +0000 UTC m=+0.047077461 container start 3eeee811732385c8771ed06d01170192de227a9a4e4cd696f379cdaeb9592022 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-6-deactivate, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, version=10, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.title=rockylinux, name=rockylinux, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.licenses=BSD-3-Clause, io.buildah.version=1.33.7, summary=Rocky Linux Base image, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.version=10, vendor=Rocky Enterprise Software Foundation, CEPH_REF=wip-pdonnell-testing-20260210.171210, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, ceph=True, license=BSD-3-Clause, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, OSD_FLAVOR=default) 2026-02-13T02:28:14.975 INFO:journalctl@ceph.osd.6.trial027.stdout:Feb 13 02:28:14 trial027 podman[82012]: 2026-02-13 02:28:14.84967797 +0000 UTC m=+0.047778703 container attach 3eeee811732385c8771ed06d01170192de227a9a4e4cd696f379cdaeb9592022 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-6-deactivate, org.opencontainers.image.licenses=BSD-3-Clause, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, ceph=True, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.documentation=https://docs.ceph.com/, license=BSD-3-Clause, summary=Rocky Linux Base image, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.version=10, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, io.buildah.version=1.33.7, name=rockylinux, CEPH_REF=wip-pdonnell-testing-20260210.171210, OSD_FLAVOR=default, org.opencontainers.image.title=rockylinux, version=10, vendor=Rocky Enterprise Software Foundation) 2026-02-13T02:28:14.975 INFO:journalctl@ceph.osd.6.trial027.stdout:Feb 13 02:28:14 trial027 podman[82012]: 2026-02-13 02:28:14.811546761 +0000 UTC m=+0.009647504 image pull b86a4a15c2dcd78db08124ee64cb48fcd681d6db032e313da19d063813b235da quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9 2026-02-13T02:28:14.975 INFO:journalctl@ceph.osd.6.trial027.stdout:Feb 13 02:28:14 trial027 podman[82012]: 2026-02-13 02:28:14.971943013 +0000 UTC m=+0.170043746 container died 3eeee811732385c8771ed06d01170192de227a9a4e4cd696f379cdaeb9592022 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-6-deactivate, vendor=Rocky Enterprise Software Foundation, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, OSD_FLAVOR=default, org.opencontainers.image.title=rockylinux, io.buildah.version=1.33.7, name=rockylinux, CEPH_REF=wip-pdonnell-testing-20260210.171210, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, summary=Rocky Linux Base image, org.opencontainers.image.version=10, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, ceph=True, license=BSD-3-Clause, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.licenses=BSD-3-Clause, version=10, org.opencontainers.image.authors=Ceph Release Team ) 2026-02-13T02:28:15.000 DEBUG:teuthology.orchestra.run.trial027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.6.service' 2026-02-13T02:28:15.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:14 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:14.666+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:15.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:14 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:14.666+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:46.873217+0000 front 2026-02-13T02:27:46.873042+0000 (oldest deadline 2026-02-13T02:28:10.972948+0000) 2026-02-13T02:28:15.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:14 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:14.666+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.073281+0000 front 2026-02-13T02:27:52.073293+0000 (oldest deadline 2026-02-13T02:28:14.373359+0000) 2026-02-13T02:28:15.036 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T02:28:15.037 INFO:tasks.cephadm.osd.6:Stopped osd.6 2026-02-13T02:28:15.037 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2026-02-13T02:28:15.037 DEBUG:teuthology.orchestra.run.trial028:> sudo systemctl stop ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.7 2026-02-13T02:28:15.493 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:15 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:15.011+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:15.493 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:15 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:15.011+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.710229+0000 front 2026-02-13T02:27:43.710211+0000 (oldest deadline 2026-02-13T02:28:07.210302+0000) 2026-02-13T02:28:15.493 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:15 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:15.011+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:51.011317+0000 front 2026-02-13T02:27:51.011373+0000 (oldest deadline 2026-02-13T02:28:14.511371+0000) 2026-02-13T02:28:15.494 INFO:journalctl@ceph.osd.7.trial028.stdout:Feb 13 02:28:15 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-7[44087]: 2026-02-13T02:28:15.009+0000 7f10ff88a6c0 -1 osd.7 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:45.387784+0000 front 2026-02-13T02:27:45.387797+0000 (oldest deadline 2026-02-13T02:28:10.687895+0000) 2026-02-13T02:28:15.494 INFO:journalctl@ceph.osd.7.trial028.stdout:Feb 13 02:28:15 trial028 systemd[1]: Stopping Ceph osd.7 for 707dbb29-0882-11f1-9262-d404e6e7d460... 2026-02-13T02:28:15.494 INFO:journalctl@ceph.osd.7.trial028.stdout:Feb 13 02:28:15 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-7[44087]: 2026-02-13T02:28:15.139+0000 7f11031c56c0 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --osd-objectstore=bluestore (PID: 1) UID: 0 2026-02-13T02:28:15.494 INFO:journalctl@ceph.osd.7.trial028.stdout:Feb 13 02:28:15 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-7[44087]: 2026-02-13T02:28:15.139+0000 7f11031c56c0 -1 osd.7 72 *** Got signal Terminated *** 2026-02-13T02:28:15.494 INFO:journalctl@ceph.osd.7.trial028.stdout:Feb 13 02:28:15 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-7[44087]: 2026-02-13T02:28:15.139+0000 7f11031c56c0 -1 osd.7 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-13T02:28:15.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:15 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:15.164+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.778102+0000 front 2026-02-13T02:27:37.778172+0000 (oldest deadline 2026-02-13T02:28:03.678195+0000) 2026-02-13T02:28:15.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:15 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:15.164+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:44.778785+0000 front 2026-02-13T02:27:44.778712+0000 (oldest deadline 2026-02-13T02:28:07.678732+0000) 2026-02-13T02:28:16.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:15 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:15.682+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:16.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:15 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:15.682+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:46.873217+0000 front 2026-02-13T02:27:46.873042+0000 (oldest deadline 2026-02-13T02:28:10.972948+0000) 2026-02-13T02:28:16.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:15 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:15.682+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.073281+0000 front 2026-02-13T02:27:52.073293+0000 (oldest deadline 2026-02-13T02:28:14.373359+0000) 2026-02-13T02:28:16.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:15 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:15.841+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:39.585067+0000 front 2026-02-13T02:27:39.585049+0000 (oldest deadline 2026-02-13T02:28:03.685105+0000) 2026-02-13T02:28:16.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:15 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:15.841+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.685350+0000 front 2026-02-13T02:27:43.685369+0000 (oldest deadline 2026-02-13T02:28:08.385425+0000) 2026-02-13T02:28:16.159 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:15 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:15.841+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.386146+0000 front 2026-02-13T02:27:52.386067+0000 (oldest deadline 2026-02-13T02:28:15.286116+0000) 2026-02-13T02:28:16.243 INFO:journalctl@ceph.osd.7.trial028.stdout:Feb 13 02:28:15 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-7[44087]: 2026-02-13T02:28:15.962+0000 7f10ff88a6c0 -1 osd.7 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:45.387784+0000 front 2026-02-13T02:27:45.387797+0000 (oldest deadline 2026-02-13T02:28:10.687895+0000) 2026-02-13T02:28:16.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:15 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:15.962+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:16.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:15 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:15.962+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.710229+0000 front 2026-02-13T02:27:43.710211+0000 (oldest deadline 2026-02-13T02:28:07.210302+0000) 2026-02-13T02:28:16.244 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:15 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:15.962+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:51.011317+0000 front 2026-02-13T02:27:51.011373+0000 (oldest deadline 2026-02-13T02:28:14.511371+0000) 2026-02-13T02:28:16.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:16 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:16.203+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.778102+0000 front 2026-02-13T02:27:37.778172+0000 (oldest deadline 2026-02-13T02:28:03.678195+0000) 2026-02-13T02:28:16.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:16 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:16.203+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:44.778785+0000 front 2026-02-13T02:27:44.778712+0000 (oldest deadline 2026-02-13T02:28:07.678732+0000) 2026-02-13T02:28:16.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:16 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:16.203+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:49.979052+0000 front 2026-02-13T02:27:49.979070+0000 (oldest deadline 2026-02-13T02:28:15.879111+0000) 2026-02-13T02:28:17.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:16 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:16.714+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:17.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:16 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:16.714+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:46.873217+0000 front 2026-02-13T02:27:46.873042+0000 (oldest deadline 2026-02-13T02:28:10.972948+0000) 2026-02-13T02:28:17.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:16 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:16.714+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.073281+0000 front 2026-02-13T02:27:52.073293+0000 (oldest deadline 2026-02-13T02:28:14.373359+0000) 2026-02-13T02:28:17.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:16 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:16.874+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:39.585067+0000 front 2026-02-13T02:27:39.585049+0000 (oldest deadline 2026-02-13T02:28:03.685105+0000) 2026-02-13T02:28:17.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:16 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:16.874+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.685350+0000 front 2026-02-13T02:27:43.685369+0000 (oldest deadline 2026-02-13T02:28:08.385425+0000) 2026-02-13T02:28:17.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:16 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:16.874+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.386146+0000 front 2026-02-13T02:27:52.386067+0000 (oldest deadline 2026-02-13T02:28:15.286116+0000) 2026-02-13T02:28:17.243 INFO:journalctl@ceph.osd.7.trial028.stdout:Feb 13 02:28:16 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-7[44087]: 2026-02-13T02:28:16.952+0000 7f10ff88a6c0 -1 osd.7 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:45.387784+0000 front 2026-02-13T02:27:45.387797+0000 (oldest deadline 2026-02-13T02:28:10.687895+0000) 2026-02-13T02:28:17.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:16 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:16.986+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:17.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:16 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:16.986+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.710229+0000 front 2026-02-13T02:27:43.710211+0000 (oldest deadline 2026-02-13T02:28:07.210302+0000) 2026-02-13T02:28:17.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:16 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:16.986+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:51.011317+0000 front 2026-02-13T02:27:51.011373+0000 (oldest deadline 2026-02-13T02:28:14.511371+0000) 2026-02-13T02:28:17.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:17 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:17.227+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.778102+0000 front 2026-02-13T02:27:37.778172+0000 (oldest deadline 2026-02-13T02:28:03.678195+0000) 2026-02-13T02:28:17.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:17 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:17.227+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:44.778785+0000 front 2026-02-13T02:27:44.778712+0000 (oldest deadline 2026-02-13T02:28:07.678732+0000) 2026-02-13T02:28:17.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:17 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:17.227+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:49.979052+0000 front 2026-02-13T02:27:49.979070+0000 (oldest deadline 2026-02-13T02:28:15.879111+0000) 2026-02-13T02:28:18.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:17 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:17.704+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:18.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:17 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:17.704+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:46.873217+0000 front 2026-02-13T02:27:46.873042+0000 (oldest deadline 2026-02-13T02:28:10.972948+0000) 2026-02-13T02:28:18.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:17 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:17.704+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.073281+0000 front 2026-02-13T02:27:52.073293+0000 (oldest deadline 2026-02-13T02:28:14.373359+0000) 2026-02-13T02:28:18.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:17 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:17.861+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:39.585067+0000 front 2026-02-13T02:27:39.585049+0000 (oldest deadline 2026-02-13T02:28:03.685105+0000) 2026-02-13T02:28:18.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:17 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:17.861+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.685350+0000 front 2026-02-13T02:27:43.685369+0000 (oldest deadline 2026-02-13T02:28:08.385425+0000) 2026-02-13T02:28:18.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:17 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:17.861+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.386146+0000 front 2026-02-13T02:27:52.386067+0000 (oldest deadline 2026-02-13T02:28:15.286116+0000) 2026-02-13T02:28:18.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:17 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:17.953+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:18.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:17 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:17.953+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.710229+0000 front 2026-02-13T02:27:43.710211+0000 (oldest deadline 2026-02-13T02:28:07.210302+0000) 2026-02-13T02:28:18.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:17 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:17.953+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:51.011317+0000 front 2026-02-13T02:27:51.011373+0000 (oldest deadline 2026-02-13T02:28:14.511371+0000) 2026-02-13T02:28:18.243 INFO:journalctl@ceph.osd.7.trial028.stdout:Feb 13 02:28:17 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-7[44087]: 2026-02-13T02:28:17.973+0000 7f10ff88a6c0 -1 osd.7 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:45.387784+0000 front 2026-02-13T02:27:45.387797+0000 (oldest deadline 2026-02-13T02:28:10.687895+0000) 2026-02-13T02:28:18.244 INFO:journalctl@ceph.osd.7.trial028.stdout:Feb 13 02:28:17 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-7[44087]: 2026-02-13T02:28:17.973+0000 7f10ff88a6c0 -1 osd.7 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:51.788414+0000 front 2026-02-13T02:27:51.788407+0000 (oldest deadline 2026-02-13T02:28:17.688407+0000) 2026-02-13T02:28:18.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:18 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:18.196+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.778102+0000 front 2026-02-13T02:27:37.778172+0000 (oldest deadline 2026-02-13T02:28:03.678195+0000) 2026-02-13T02:28:18.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:18 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:18.196+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:44.778785+0000 front 2026-02-13T02:27:44.778712+0000 (oldest deadline 2026-02-13T02:28:07.678732+0000) 2026-02-13T02:28:18.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:18 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:18.196+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:49.979052+0000 front 2026-02-13T02:27:49.979070+0000 (oldest deadline 2026-02-13T02:28:15.879111+0000) 2026-02-13T02:28:18.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:18 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:18.196+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:56.979590+0000 front 2026-02-13T02:27:56.979568+0000 (oldest deadline 2026-02-13T02:28:18.079604+0000) 2026-02-13T02:28:19.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:18 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:18.728+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:19.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:18 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:18.728+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:46.873217+0000 front 2026-02-13T02:27:46.873042+0000 (oldest deadline 2026-02-13T02:28:10.972948+0000) 2026-02-13T02:28:19.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:18 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:18.728+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.073281+0000 front 2026-02-13T02:27:52.073293+0000 (oldest deadline 2026-02-13T02:28:14.373359+0000) 2026-02-13T02:28:19.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:18 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:18.850+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:39.585067+0000 front 2026-02-13T02:27:39.585049+0000 (oldest deadline 2026-02-13T02:28:03.685105+0000) 2026-02-13T02:28:19.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:18 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:18.850+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.685350+0000 front 2026-02-13T02:27:43.685369+0000 (oldest deadline 2026-02-13T02:28:08.385425+0000) 2026-02-13T02:28:19.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:18 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:18.850+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.386146+0000 front 2026-02-13T02:27:52.386067+0000 (oldest deadline 2026-02-13T02:28:15.286116+0000) 2026-02-13T02:28:19.243 INFO:journalctl@ceph.osd.7.trial028.stdout:Feb 13 02:28:18 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-7[44087]: 2026-02-13T02:28:18.999+0000 7f10ff88a6c0 -1 osd.7 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:45.387784+0000 front 2026-02-13T02:27:45.387797+0000 (oldest deadline 2026-02-13T02:28:10.687895+0000) 2026-02-13T02:28:19.243 INFO:journalctl@ceph.osd.7.trial028.stdout:Feb 13 02:28:18 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-7[44087]: 2026-02-13T02:28:18.999+0000 7f10ff88a6c0 -1 osd.7 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:51.788414+0000 front 2026-02-13T02:27:51.788407+0000 (oldest deadline 2026-02-13T02:28:17.688407+0000) 2026-02-13T02:28:19.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:18 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:18.917+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:19.244 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:18 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:18.917+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.710229+0000 front 2026-02-13T02:27:43.710211+0000 (oldest deadline 2026-02-13T02:28:07.210302+0000) 2026-02-13T02:28:19.244 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:18 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:18.917+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:51.011317+0000 front 2026-02-13T02:27:51.011373+0000 (oldest deadline 2026-02-13T02:28:14.511371+0000) 2026-02-13T02:28:19.244 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:18 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:18.917+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:54.511577+0000 front 2026-02-13T02:27:54.511589+0000 (oldest deadline 2026-02-13T02:28:18.611662+0000) 2026-02-13T02:28:19.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:19 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:19.208+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.778102+0000 front 2026-02-13T02:27:37.778172+0000 (oldest deadline 2026-02-13T02:28:03.678195+0000) 2026-02-13T02:28:19.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:19 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:19.208+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:44.778785+0000 front 2026-02-13T02:27:44.778712+0000 (oldest deadline 2026-02-13T02:28:07.678732+0000) 2026-02-13T02:28:19.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:19 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:19.208+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:49.979052+0000 front 2026-02-13T02:27:49.979070+0000 (oldest deadline 2026-02-13T02:28:15.879111+0000) 2026-02-13T02:28:19.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:19 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:19.208+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:56.979590+0000 front 2026-02-13T02:27:56.979568+0000 (oldest deadline 2026-02-13T02:28:18.079604+0000) 2026-02-13T02:28:20.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:19 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:19.726+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:20.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:19 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:19.726+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:46.873217+0000 front 2026-02-13T02:27:46.873042+0000 (oldest deadline 2026-02-13T02:28:10.972948+0000) 2026-02-13T02:28:20.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:19 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:19.726+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.073281+0000 front 2026-02-13T02:27:52.073293+0000 (oldest deadline 2026-02-13T02:28:14.373359+0000) 2026-02-13T02:28:20.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:19 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:19.827+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:39.585067+0000 front 2026-02-13T02:27:39.585049+0000 (oldest deadline 2026-02-13T02:28:03.685105+0000) 2026-02-13T02:28:20.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:19 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:19.827+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.685350+0000 front 2026-02-13T02:27:43.685369+0000 (oldest deadline 2026-02-13T02:28:08.385425+0000) 2026-02-13T02:28:20.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:19 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:19.827+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.386146+0000 front 2026-02-13T02:27:52.386067+0000 (oldest deadline 2026-02-13T02:28:15.286116+0000) 2026-02-13T02:28:20.206 INFO:journalctl@ceph.osd.7.trial028.stdout:Feb 13 02:28:20 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-7[44087]: 2026-02-13T02:28:20.035+0000 7f10ff88a6c0 -1 osd.7 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:45.387784+0000 front 2026-02-13T02:27:45.387797+0000 (oldest deadline 2026-02-13T02:28:10.687895+0000) 2026-02-13T02:28:20.206 INFO:journalctl@ceph.osd.7.trial028.stdout:Feb 13 02:28:20 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-7[44087]: 2026-02-13T02:28:20.035+0000 7f10ff88a6c0 -1 osd.7 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:51.788414+0000 front 2026-02-13T02:27:51.788407+0000 (oldest deadline 2026-02-13T02:28:17.688407+0000) 2026-02-13T02:28:20.206 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:19 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:19.957+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:20.207 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:19 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:19.957+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.710229+0000 front 2026-02-13T02:27:43.710211+0000 (oldest deadline 2026-02-13T02:28:07.210302+0000) 2026-02-13T02:28:20.207 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:19 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:19.957+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:51.011317+0000 front 2026-02-13T02:27:51.011373+0000 (oldest deadline 2026-02-13T02:28:14.511371+0000) 2026-02-13T02:28:20.207 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:19 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:19.957+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:54.511577+0000 front 2026-02-13T02:27:54.511589+0000 (oldest deadline 2026-02-13T02:28:18.611662+0000) 2026-02-13T02:28:20.494 INFO:journalctl@ceph.osd.7.trial028.stdout:Feb 13 02:28:20 trial028 podman[58127]: 2026-02-13 02:28:20.209036267 +0000 UTC m=+5.083000236 container died 774605533e8664a5c7bea94aebc5e3158484c54847b9a555b981942d557c4a9a (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-7, org.opencontainers.image.version=10, org.opencontainers.image.documentation=https://docs.ceph.com/, license=BSD-3-Clause, CEPH_REF=wip-pdonnell-testing-20260210.171210, ceph=True, summary=Rocky Linux Base image, org.opencontainers.image.licenses=BSD-3-Clause, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.authors=Ceph Release Team , vendor=Rocky Enterprise Software Foundation, OSD_FLAVOR=default, io.buildah.version=1.33.7, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, version=10, org.opencontainers.image.title=rockylinux, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, name=rockylinux, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation) 2026-02-13T02:28:20.494 INFO:journalctl@ceph.osd.7.trial028.stdout:Feb 13 02:28:20 trial028 podman[58127]: 2026-02-13 02:28:20.217459756 +0000 UTC m=+5.091423715 container remove 774605533e8664a5c7bea94aebc5e3158484c54847b9a555b981942d557c4a9a (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-7, ceph=True, vendor=Rocky Enterprise Software Foundation, io.buildah.version=1.33.7, org.opencontainers.image.documentation=https://docs.ceph.com/, OSD_FLAVOR=default, version=10, name=rockylinux, org.opencontainers.image.title=rockylinux, CEPH_REF=wip-pdonnell-testing-20260210.171210, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, license=BSD-3-Clause, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, org.opencontainers.image.licenses=BSD-3-Clause, org.opencontainers.image.authors=Ceph Release Team , summary=Rocky Linux Base image, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.version=10, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f) 2026-02-13T02:28:20.494 INFO:journalctl@ceph.osd.7.trial028.stdout:Feb 13 02:28:20 trial028 bash[58127]: ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-7 2026-02-13T02:28:20.494 INFO:journalctl@ceph.osd.7.trial028.stdout:Feb 13 02:28:20 trial028 podman[58255]: 2026-02-13 02:28:20.351147149 +0000 UTC m=+0.016613575 container create 2eaba422900eab2e8bb21b12f4478937bd839fb219a042c7f427d08463597f67 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-7-deactivate, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.version=10, org.opencontainers.image.authors=Ceph Release Team , io.buildah.version=1.33.7, OSD_FLAVOR=default, org.opencontainers.image.title=rockylinux, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.licenses=BSD-3-Clause, CEPH_REF=wip-pdonnell-testing-20260210.171210, summary=Rocky Linux Base image, name=rockylinux, license=BSD-3-Clause, ceph=True, version=10, org.opencontainers.image.documentation=https://docs.ceph.com/) 2026-02-13T02:28:20.494 INFO:journalctl@ceph.osd.7.trial028.stdout:Feb 13 02:28:20 trial028 podman[58255]: 2026-02-13 02:28:20.381275626 +0000 UTC m=+0.046741992 container init 2eaba422900eab2e8bb21b12f4478937bd839fb219a042c7f427d08463597f67 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-7-deactivate, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, OSD_FLAVOR=default, name=rockylinux, org.opencontainers.image.authors=Ceph Release Team , GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, version=10, summary=Rocky Linux Base image, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, license=BSD-3-Clause, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, vendor=Rocky Enterprise Software Foundation, CEPH_REF=wip-pdonnell-testing-20260210.171210, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.title=rockylinux, io.buildah.version=1.33.7, org.opencontainers.image.licenses=BSD-3-Clause, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, org.opencontainers.image.version=10, ceph=True) 2026-02-13T02:28:20.494 INFO:journalctl@ceph.osd.7.trial028.stdout:Feb 13 02:28:20 trial028 podman[58255]: 2026-02-13 02:28:20.383523158 +0000 UTC m=+0.048989504 container start 2eaba422900eab2e8bb21b12f4478937bd839fb219a042c7f427d08463597f67 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-7-deactivate, org.opencontainers.image.licenses=BSD-3-Clause, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, CEPH_REF=wip-pdonnell-testing-20260210.171210, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, OSD_FLAVOR=default, name=rockylinux, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.title=rockylinux, org.opencontainers.image.authors=Ceph Release Team , license=BSD-3-Clause, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, vendor=Rocky Enterprise Software Foundation, version=10, summary=Rocky Linux Base image, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.version=10, ceph=True, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, io.buildah.version=1.33.7) 2026-02-13T02:28:20.494 INFO:journalctl@ceph.osd.7.trial028.stdout:Feb 13 02:28:20 trial028 podman[58255]: 2026-02-13 02:28:20.384025641 +0000 UTC m=+0.049491997 container attach 2eaba422900eab2e8bb21b12f4478937bd839fb219a042c7f427d08463597f67 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-7-deactivate, org.opencontainers.image.authors=Ceph Release Team , GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_REF=wip-pdonnell-testing-20260210.171210, version=10, name=rockylinux, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, OSD_FLAVOR=default, summary=Rocky Linux Base image, org.opencontainers.image.licenses=BSD-3-Clause, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, ceph=True, io.buildah.version=1.33.7, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.version=10, org.opencontainers.image.title=rockylinux, license=BSD-3-Clause) 2026-02-13T02:28:20.494 INFO:journalctl@ceph.osd.7.trial028.stdout:Feb 13 02:28:20 trial028 podman[58255]: 2026-02-13 02:28:20.344521759 +0000 UTC m=+0.009988125 image pull b86a4a15c2dcd78db08124ee64cb48fcd681d6db032e313da19d063813b235da quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9 2026-02-13T02:28:20.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:20 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:20.218+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.778102+0000 front 2026-02-13T02:27:37.778172+0000 (oldest deadline 2026-02-13T02:28:03.678195+0000) 2026-02-13T02:28:20.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:20 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:20.218+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:44.778785+0000 front 2026-02-13T02:27:44.778712+0000 (oldest deadline 2026-02-13T02:28:07.678732+0000) 2026-02-13T02:28:20.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:20 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:20.218+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:49.979052+0000 front 2026-02-13T02:27:49.979070+0000 (oldest deadline 2026-02-13T02:28:15.879111+0000) 2026-02-13T02:28:20.501 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:20 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:20.218+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:56.979590+0000 front 2026-02-13T02:27:56.979568+0000 (oldest deadline 2026-02-13T02:28:18.079604+0000) 2026-02-13T02:28:20.537 DEBUG:teuthology.orchestra.run.trial028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.7.service' 2026-02-13T02:28:20.573 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T02:28:20.574 INFO:tasks.cephadm.osd.7:Stopped osd.7 2026-02-13T02:28:20.574 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2026-02-13T02:28:20.574 DEBUG:teuthology.orchestra.run.trial107:> sudo systemctl stop ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.8 2026-02-13T02:28:21.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:20 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:20.684+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:21.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:20 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:20.684+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:46.873217+0000 front 2026-02-13T02:27:46.873042+0000 (oldest deadline 2026-02-13T02:28:10.972948+0000) 2026-02-13T02:28:21.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:20 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:20.684+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.073281+0000 front 2026-02-13T02:27:52.073293+0000 (oldest deadline 2026-02-13T02:28:14.373359+0000) 2026-02-13T02:28:21.002 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:20 trial107 systemd[1]: Stopping Ceph osd.8 for 707dbb29-0882-11f1-9262-d404e6e7d460... 2026-02-13T02:28:21.002 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:20 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:20.677+0000 7f97d8b706c0 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --osd-objectstore=bluestore (PID: 1) UID: 0 2026-02-13T02:28:21.002 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:20 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:20.677+0000 7f97d8b706c0 -1 osd.8 72 *** Got signal Terminated *** 2026-02-13T02:28:21.002 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:20 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:20.677+0000 7f97d8b706c0 -1 osd.8 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-13T02:28:21.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:20 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:20.810+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:39.585067+0000 front 2026-02-13T02:27:39.585049+0000 (oldest deadline 2026-02-13T02:28:03.685105+0000) 2026-02-13T02:28:21.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:20 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:20.810+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.685350+0000 front 2026-02-13T02:27:43.685369+0000 (oldest deadline 2026-02-13T02:28:08.385425+0000) 2026-02-13T02:28:21.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:20 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:20.810+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.386146+0000 front 2026-02-13T02:27:52.386067+0000 (oldest deadline 2026-02-13T02:28:15.286116+0000) 2026-02-13T02:28:21.159 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:20 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:20.810+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:55.286247+0000 front 2026-02-13T02:27:55.286269+0000 (oldest deadline 2026-02-13T02:28:20.586295+0000) 2026-02-13T02:28:21.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:20 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:20.939+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:21.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:20 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:20.939+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.710229+0000 front 2026-02-13T02:27:43.710211+0000 (oldest deadline 2026-02-13T02:28:07.210302+0000) 2026-02-13T02:28:21.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:20 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:20.939+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:51.011317+0000 front 2026-02-13T02:27:51.011373+0000 (oldest deadline 2026-02-13T02:28:14.511371+0000) 2026-02-13T02:28:21.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:20 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:20.939+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:54.511577+0000 front 2026-02-13T02:27:54.511589+0000 (oldest deadline 2026-02-13T02:28:18.611662+0000) 2026-02-13T02:28:21.660 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:21 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:21.266+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.778102+0000 front 2026-02-13T02:27:37.778172+0000 (oldest deadline 2026-02-13T02:28:03.678195+0000) 2026-02-13T02:28:21.660 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:21 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:21.266+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:44.778785+0000 front 2026-02-13T02:27:44.778712+0000 (oldest deadline 2026-02-13T02:28:07.678732+0000) 2026-02-13T02:28:21.660 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:21 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:21.266+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:49.979052+0000 front 2026-02-13T02:27:49.979070+0000 (oldest deadline 2026-02-13T02:28:15.879111+0000) 2026-02-13T02:28:21.660 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:21 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:21.266+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:56.979590+0000 front 2026-02-13T02:27:56.979568+0000 (oldest deadline 2026-02-13T02:28:18.079604+0000) 2026-02-13T02:28:22.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:21 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:21.662+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:22.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:21 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:21.662+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:46.873217+0000 front 2026-02-13T02:27:46.873042+0000 (oldest deadline 2026-02-13T02:28:10.972948+0000) 2026-02-13T02:28:22.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:21 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:21.662+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.073281+0000 front 2026-02-13T02:27:52.073293+0000 (oldest deadline 2026-02-13T02:28:14.373359+0000) 2026-02-13T02:28:22.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:21 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:21.830+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:39.585067+0000 front 2026-02-13T02:27:39.585049+0000 (oldest deadline 2026-02-13T02:28:03.685105+0000) 2026-02-13T02:28:22.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:21 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:21.830+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.685350+0000 front 2026-02-13T02:27:43.685369+0000 (oldest deadline 2026-02-13T02:28:08.385425+0000) 2026-02-13T02:28:22.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:21 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:21.830+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.386146+0000 front 2026-02-13T02:27:52.386067+0000 (oldest deadline 2026-02-13T02:28:15.286116+0000) 2026-02-13T02:28:22.159 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:21 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:21.830+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:55.286247+0000 front 2026-02-13T02:27:55.286269+0000 (oldest deadline 2026-02-13T02:28:20.586295+0000) 2026-02-13T02:28:22.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:21 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:21.938+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:22.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:21 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:21.938+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.710229+0000 front 2026-02-13T02:27:43.710211+0000 (oldest deadline 2026-02-13T02:28:07.210302+0000) 2026-02-13T02:28:22.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:21 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:21.938+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:51.011317+0000 front 2026-02-13T02:27:51.011373+0000 (oldest deadline 2026-02-13T02:28:14.511371+0000) 2026-02-13T02:28:22.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:21 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:21.938+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:54.511577+0000 front 2026-02-13T02:27:54.511589+0000 (oldest deadline 2026-02-13T02:28:18.611662+0000) 2026-02-13T02:28:22.668 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:22 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:22.300+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.778102+0000 front 2026-02-13T02:27:37.778172+0000 (oldest deadline 2026-02-13T02:28:03.678195+0000) 2026-02-13T02:28:22.668 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:22 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:22.300+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:44.778785+0000 front 2026-02-13T02:27:44.778712+0000 (oldest deadline 2026-02-13T02:28:07.678732+0000) 2026-02-13T02:28:22.668 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:22 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:22.300+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:49.979052+0000 front 2026-02-13T02:27:49.979070+0000 (oldest deadline 2026-02-13T02:28:15.879111+0000) 2026-02-13T02:28:22.668 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:22 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:22.300+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:56.979590+0000 front 2026-02-13T02:27:56.979568+0000 (oldest deadline 2026-02-13T02:28:18.079604+0000) 2026-02-13T02:28:23.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:22 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:22.669+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:23.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:22 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:22.669+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:46.873217+0000 front 2026-02-13T02:27:46.873042+0000 (oldest deadline 2026-02-13T02:28:10.972948+0000) 2026-02-13T02:28:23.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:22 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:22.669+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.073281+0000 front 2026-02-13T02:27:52.073293+0000 (oldest deadline 2026-02-13T02:28:14.373359+0000) 2026-02-13T02:28:23.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:22 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:22.800+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:39.585067+0000 front 2026-02-13T02:27:39.585049+0000 (oldest deadline 2026-02-13T02:28:03.685105+0000) 2026-02-13T02:28:23.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:22 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:22.800+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.685350+0000 front 2026-02-13T02:27:43.685369+0000 (oldest deadline 2026-02-13T02:28:08.385425+0000) 2026-02-13T02:28:23.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:22 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:22.800+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.386146+0000 front 2026-02-13T02:27:52.386067+0000 (oldest deadline 2026-02-13T02:28:15.286116+0000) 2026-02-13T02:28:23.159 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:22 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:22.800+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:55.286247+0000 front 2026-02-13T02:27:55.286269+0000 (oldest deadline 2026-02-13T02:28:20.586295+0000) 2026-02-13T02:28:23.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:22 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:22.936+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:23.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:22 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:22.936+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.710229+0000 front 2026-02-13T02:27:43.710211+0000 (oldest deadline 2026-02-13T02:28:07.210302+0000) 2026-02-13T02:28:23.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:22 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:22.936+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:51.011317+0000 front 2026-02-13T02:27:51.011373+0000 (oldest deadline 2026-02-13T02:28:14.511371+0000) 2026-02-13T02:28:23.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:22 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:22.936+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:54.511577+0000 front 2026-02-13T02:27:54.511589+0000 (oldest deadline 2026-02-13T02:28:18.611662+0000) 2026-02-13T02:28:23.690 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:23 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:23.308+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.778102+0000 front 2026-02-13T02:27:37.778172+0000 (oldest deadline 2026-02-13T02:28:03.678195+0000) 2026-02-13T02:28:23.690 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:23 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:23.308+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:44.778785+0000 front 2026-02-13T02:27:44.778712+0000 (oldest deadline 2026-02-13T02:28:07.678732+0000) 2026-02-13T02:28:23.690 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:23 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:23.308+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:49.979052+0000 front 2026-02-13T02:27:49.979070+0000 (oldest deadline 2026-02-13T02:28:15.879111+0000) 2026-02-13T02:28:23.691 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:23 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:23.308+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:56.979590+0000 front 2026-02-13T02:27:56.979568+0000 (oldest deadline 2026-02-13T02:28:18.079604+0000) 2026-02-13T02:28:24.000 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:23 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:23.692+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:24.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:23 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:23.692+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:46.873217+0000 front 2026-02-13T02:27:46.873042+0000 (oldest deadline 2026-02-13T02:28:10.972948+0000) 2026-02-13T02:28:24.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:23 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:23.692+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.073281+0000 front 2026-02-13T02:27:52.073293+0000 (oldest deadline 2026-02-13T02:28:14.373359+0000) 2026-02-13T02:28:24.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:23 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:23.776+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:39.585067+0000 front 2026-02-13T02:27:39.585049+0000 (oldest deadline 2026-02-13T02:28:03.685105+0000) 2026-02-13T02:28:24.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:23 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:23.776+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.685350+0000 front 2026-02-13T02:27:43.685369+0000 (oldest deadline 2026-02-13T02:28:08.385425+0000) 2026-02-13T02:28:24.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:23 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:23.776+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.386146+0000 front 2026-02-13T02:27:52.386067+0000 (oldest deadline 2026-02-13T02:28:15.286116+0000) 2026-02-13T02:28:24.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:23 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:23.776+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:55.286247+0000 front 2026-02-13T02:27:55.286269+0000 (oldest deadline 2026-02-13T02:28:20.586295+0000) 2026-02-13T02:28:24.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:23 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:23.973+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:24.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:23 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:23.973+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.710229+0000 front 2026-02-13T02:27:43.710211+0000 (oldest deadline 2026-02-13T02:28:07.210302+0000) 2026-02-13T02:28:24.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:23 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:23.973+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:51.011317+0000 front 2026-02-13T02:27:51.011373+0000 (oldest deadline 2026-02-13T02:28:14.511371+0000) 2026-02-13T02:28:24.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:23 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:23.973+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:54.511577+0000 front 2026-02-13T02:27:54.511589+0000 (oldest deadline 2026-02-13T02:28:18.611662+0000) 2026-02-13T02:28:24.665 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:24 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:24.320+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.778102+0000 front 2026-02-13T02:27:37.778172+0000 (oldest deadline 2026-02-13T02:28:03.678195+0000) 2026-02-13T02:28:24.665 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:24 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:24.320+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:44.778785+0000 front 2026-02-13T02:27:44.778712+0000 (oldest deadline 2026-02-13T02:28:07.678732+0000) 2026-02-13T02:28:24.665 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:24 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:24.320+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:49.979052+0000 front 2026-02-13T02:27:49.979070+0000 (oldest deadline 2026-02-13T02:28:15.879111+0000) 2026-02-13T02:28:24.665 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:24 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:24.320+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:56.979590+0000 front 2026-02-13T02:27:56.979568+0000 (oldest deadline 2026-02-13T02:28:18.079604+0000) 2026-02-13T02:28:24.666 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:24 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:24.320+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:27:58.079933+0000 front 2026-02-13T02:27:58.079989+0000 (oldest deadline 2026-02-13T02:28:23.979827+0000) 2026-02-13T02:28:25.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:24 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:24.667+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:25.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:24 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:24.667+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:46.873217+0000 front 2026-02-13T02:27:46.873042+0000 (oldest deadline 2026-02-13T02:28:10.972948+0000) 2026-02-13T02:28:25.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:24 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:24.667+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.073281+0000 front 2026-02-13T02:27:52.073293+0000 (oldest deadline 2026-02-13T02:28:14.373359+0000) 2026-02-13T02:28:25.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:24 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:24.667+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:27:59.073720+0000 front 2026-02-13T02:27:59.073711+0000 (oldest deadline 2026-02-13T02:28:23.773773+0000) 2026-02-13T02:28:25.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:24 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:24.775+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:39.585067+0000 front 2026-02-13T02:27:39.585049+0000 (oldest deadline 2026-02-13T02:28:03.685105+0000) 2026-02-13T02:28:25.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:24 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:24.775+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.685350+0000 front 2026-02-13T02:27:43.685369+0000 (oldest deadline 2026-02-13T02:28:08.385425+0000) 2026-02-13T02:28:25.159 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:24 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:24.775+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.386146+0000 front 2026-02-13T02:27:52.386067+0000 (oldest deadline 2026-02-13T02:28:15.286116+0000) 2026-02-13T02:28:25.159 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:24 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:24.775+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:55.286247+0000 front 2026-02-13T02:27:55.286269+0000 (oldest deadline 2026-02-13T02:28:20.586295+0000) 2026-02-13T02:28:25.159 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:24 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:24.775+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:28:02.286791+0000 front 2026-02-13T02:28:02.286705+0000 (oldest deadline 2026-02-13T02:28:23.986736+0000) 2026-02-13T02:28:25.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:24 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:24.976+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:25.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:24 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:24.976+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.710229+0000 front 2026-02-13T02:27:43.710211+0000 (oldest deadline 2026-02-13T02:28:07.210302+0000) 2026-02-13T02:28:25.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:24 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:24.976+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:51.011317+0000 front 2026-02-13T02:27:51.011373+0000 (oldest deadline 2026-02-13T02:28:14.511371+0000) 2026-02-13T02:28:25.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:24 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:24.976+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:54.511577+0000 front 2026-02-13T02:27:54.511589+0000 (oldest deadline 2026-02-13T02:28:18.611662+0000) 2026-02-13T02:28:25.695 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:25 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:25.322+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:37.778102+0000 front 2026-02-13T02:27:37.778172+0000 (oldest deadline 2026-02-13T02:28:03.678195+0000) 2026-02-13T02:28:25.695 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:25 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:25.322+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:44.778785+0000 front 2026-02-13T02:27:44.778712+0000 (oldest deadline 2026-02-13T02:28:07.678732+0000) 2026-02-13T02:28:25.695 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:25 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:25.322+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:49.979052+0000 front 2026-02-13T02:27:49.979070+0000 (oldest deadline 2026-02-13T02:28:15.879111+0000) 2026-02-13T02:28:25.696 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:25 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:25.322+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:56.979590+0000 front 2026-02-13T02:27:56.979568+0000 (oldest deadline 2026-02-13T02:28:18.079604+0000) 2026-02-13T02:28:25.696 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:25 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8[45002]: 2026-02-13T02:28:25.322+0000 7f97d52356c0 -1 osd.8 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:27:58.079933+0000 front 2026-02-13T02:27:58.079989+0000 (oldest deadline 2026-02-13T02:28:23.979827+0000) 2026-02-13T02:28:25.976 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:25 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:25.697+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:25.976 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:25 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:25.697+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:46.873217+0000 front 2026-02-13T02:27:46.873042+0000 (oldest deadline 2026-02-13T02:28:10.972948+0000) 2026-02-13T02:28:25.976 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:25 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:25.697+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.073281+0000 front 2026-02-13T02:27:52.073293+0000 (oldest deadline 2026-02-13T02:28:14.373359+0000) 2026-02-13T02:28:25.976 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:25 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:25.697+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:27:59.073720+0000 front 2026-02-13T02:27:59.073711+0000 (oldest deadline 2026-02-13T02:28:23.773773+0000) 2026-02-13T02:28:25.977 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:25 trial107 podman[60699]: 2026-02-13 02:28:25.747053847 +0000 UTC m=+5.082551059 container died 1884038d925cd456cb4d756efc1ed14a4d91ad91a381f6d6951a204de248eb07 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8, io.buildah.version=1.33.7, org.opencontainers.image.documentation=https://docs.ceph.com/, license=BSD-3-Clause, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.licenses=BSD-3-Clause, org.opencontainers.image.version=10, version=10, summary=Rocky Linux Base image, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, OSD_FLAVOR=default, org.opencontainers.image.title=rockylinux, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, ceph=True, CEPH_REF=wip-pdonnell-testing-20260210.171210, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, name=rockylinux, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/) 2026-02-13T02:28:25.977 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:25 trial107 podman[60699]: 2026-02-13 02:28:25.75614929 +0000 UTC m=+5.091646502 container remove 1884038d925cd456cb4d756efc1ed14a4d91ad91a381f6d6951a204de248eb07 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, version=10, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, license=BSD-3-Clause, name=rockylinux, ceph=True, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.licenses=BSD-3-Clause, org.opencontainers.image.title=rockylinux, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_REF=wip-pdonnell-testing-20260210.171210, org.opencontainers.image.version=10, summary=Rocky Linux Base image, vendor=Rocky Enterprise Software Foundation, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, OSD_FLAVOR=default, io.buildah.version=1.33.7) 2026-02-13T02:28:25.977 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:25 trial107 bash[60699]: ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8 2026-02-13T02:28:25.977 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:25 trial107 podman[60830]: 2026-02-13 02:28:25.885478981 +0000 UTC m=+0.014744887 container create a17c555db35f28200b043e75e01a543779f8c41fd2b45f3ca3d97f2df500ef7f (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8-deactivate, org.opencontainers.image.authors=Ceph Release Team , name=rockylinux, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, version=10, OSD_FLAVOR=default, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, summary=Rocky Linux Base image, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.licenses=BSD-3-Clause, io.buildah.version=1.33.7, org.opencontainers.image.version=10, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, org.opencontainers.image.title=rockylinux, CEPH_REF=wip-pdonnell-testing-20260210.171210, ceph=True, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.documentation=https://docs.ceph.com/, license=BSD-3-Clause) 2026-02-13T02:28:25.977 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:25 trial107 podman[60830]: 2026-02-13 02:28:25.920248768 +0000 UTC m=+0.049514674 container init a17c555db35f28200b043e75e01a543779f8c41fd2b45f3ca3d97f2df500ef7f (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8-deactivate, io.buildah.version=1.33.7, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, summary=Rocky Linux Base image, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, OSD_FLAVOR=default, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, version=10, license=BSD-3-Clause, org.opencontainers.image.licenses=BSD-3-Clause, name=rockylinux, org.opencontainers.image.version=10, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, ceph=True, vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, org.opencontainers.image.title=rockylinux, CEPH_REF=wip-pdonnell-testing-20260210.171210, org.opencontainers.image.authors=Ceph Release Team ) 2026-02-13T02:28:25.977 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:25 trial107 podman[60830]: 2026-02-13 02:28:25.922811007 +0000 UTC m=+0.052076913 container start a17c555db35f28200b043e75e01a543779f8c41fd2b45f3ca3d97f2df500ef7f (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8-deactivate, summary=Rocky Linux Base image, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, org.opencontainers.image.version=10, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, org.opencontainers.image.title=rockylinux, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.licenses=BSD-3-Clause, vendor=Rocky Enterprise Software Foundation, license=BSD-3-Clause, org.opencontainers.image.authors=Ceph Release Team , OSD_FLAVOR=default, org.opencontainers.image.documentation=https://docs.ceph.com/, version=10, CEPH_REF=wip-pdonnell-testing-20260210.171210, ceph=True, io.buildah.version=1.33.7, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, name=rockylinux, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git) 2026-02-13T02:28:25.977 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:25 trial107 podman[60830]: 2026-02-13 02:28:25.923450594 +0000 UTC m=+0.052716500 container attach a17c555db35f28200b043e75e01a543779f8c41fd2b45f3ca3d97f2df500ef7f (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-8-deactivate, OSD_FLAVOR=default, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.version=10, vendor=Rocky Enterprise Software Foundation, name=rockylinux, summary=Rocky Linux Base image, ceph=True, io.buildah.version=1.33.7, org.opencontainers.image.licenses=BSD-3-Clause, org.opencontainers.image.authors=Ceph Release Team , FROM_IMAGE=docker.io/rockylinux/rockylinux:10, org.opencontainers.image.title=rockylinux, version=10, license=BSD-3-Clause, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_REF=wip-pdonnell-testing-20260210.171210, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10) 2026-02-13T02:28:25.977 INFO:journalctl@ceph.osd.8.trial107.stdout:Feb 13 02:28:25 trial107 podman[60830]: 2026-02-13 02:28:25.879524492 +0000 UTC m=+0.008790408 image pull b86a4a15c2dcd78db08124ee64cb48fcd681d6db032e313da19d063813b235da quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9 2026-02-13T02:28:26.079 DEBUG:teuthology.orchestra.run.trial107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.8.service' 2026-02-13T02:28:26.116 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T02:28:26.117 INFO:tasks.cephadm.osd.8:Stopped osd.8 2026-02-13T02:28:26.117 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2026-02-13T02:28:26.117 DEBUG:teuthology.orchestra.run.trial027:> sudo systemctl stop ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.9 2026-02-13T02:28:26.141 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:25 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:25.808+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:39.585067+0000 front 2026-02-13T02:27:39.585049+0000 (oldest deadline 2026-02-13T02:28:03.685105+0000) 2026-02-13T02:28:26.142 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:25 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:25.808+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.685350+0000 front 2026-02-13T02:27:43.685369+0000 (oldest deadline 2026-02-13T02:28:08.385425+0000) 2026-02-13T02:28:26.142 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:25 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:25.808+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.386146+0000 front 2026-02-13T02:27:52.386067+0000 (oldest deadline 2026-02-13T02:28:15.286116+0000) 2026-02-13T02:28:26.142 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:25 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:25.808+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:55.286247+0000 front 2026-02-13T02:27:55.286269+0000 (oldest deadline 2026-02-13T02:28:20.586295+0000) 2026-02-13T02:28:26.142 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:25 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:25.808+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:28:02.286791+0000 front 2026-02-13T02:28:02.286705+0000 (oldest deadline 2026-02-13T02:28:23.986736+0000) 2026-02-13T02:28:26.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:25 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:25.994+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:26.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:25 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:25.994+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.710229+0000 front 2026-02-13T02:27:43.710211+0000 (oldest deadline 2026-02-13T02:28:07.210302+0000) 2026-02-13T02:28:26.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:25 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:25.994+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:51.011317+0000 front 2026-02-13T02:27:51.011373+0000 (oldest deadline 2026-02-13T02:28:14.511371+0000) 2026-02-13T02:28:26.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:25 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:25.994+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:54.511577+0000 front 2026-02-13T02:27:54.511589+0000 (oldest deadline 2026-02-13T02:28:18.611662+0000) 2026-02-13T02:28:26.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:25 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:25.994+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:28:00.311981+0000 front 2026-02-13T02:28:00.311999+0000 (oldest deadline 2026-02-13T02:28:25.012036+0000) 2026-02-13T02:28:26.408 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:26 trial027 systemd[1]: Stopping Ceph osd.9 for 707dbb29-0882-11f1-9262-d404e6e7d460... 2026-02-13T02:28:26.409 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:26 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:26.211+0000 7f2e28a396c0 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --osd-objectstore=bluestore (PID: 1) UID: 0 2026-02-13T02:28:26.409 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:26 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:26.211+0000 7f2e28a396c0 -1 osd.9 72 *** Got signal Terminated *** 2026-02-13T02:28:26.409 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:26 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:26.211+0000 7f2e28a396c0 -1 osd.9 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-13T02:28:27.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:26 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:26.722+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:27.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:26 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:26.722+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:46.873217+0000 front 2026-02-13T02:27:46.873042+0000 (oldest deadline 2026-02-13T02:28:10.972948+0000) 2026-02-13T02:28:27.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:26 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:26.722+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.073281+0000 front 2026-02-13T02:27:52.073293+0000 (oldest deadline 2026-02-13T02:28:14.373359+0000) 2026-02-13T02:28:27.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:26 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:26.722+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:27:59.073720+0000 front 2026-02-13T02:27:59.073711+0000 (oldest deadline 2026-02-13T02:28:23.773773+0000) 2026-02-13T02:28:27.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:26 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:26.788+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:39.585067+0000 front 2026-02-13T02:27:39.585049+0000 (oldest deadline 2026-02-13T02:28:03.685105+0000) 2026-02-13T02:28:27.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:26 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:26.788+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.685350+0000 front 2026-02-13T02:27:43.685369+0000 (oldest deadline 2026-02-13T02:28:08.385425+0000) 2026-02-13T02:28:27.159 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:26 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:26.788+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.386146+0000 front 2026-02-13T02:27:52.386067+0000 (oldest deadline 2026-02-13T02:28:15.286116+0000) 2026-02-13T02:28:27.159 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:26 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:26.788+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:55.286247+0000 front 2026-02-13T02:27:55.286269+0000 (oldest deadline 2026-02-13T02:28:20.586295+0000) 2026-02-13T02:28:27.159 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:26 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:26.788+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:28:02.286791+0000 front 2026-02-13T02:28:02.286705+0000 (oldest deadline 2026-02-13T02:28:23.986736+0000) 2026-02-13T02:28:27.493 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:27 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:26.999+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:27.493 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:27 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:26.999+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.710229+0000 front 2026-02-13T02:27:43.710211+0000 (oldest deadline 2026-02-13T02:28:07.210302+0000) 2026-02-13T02:28:27.493 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:27 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:26.999+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:51.011317+0000 front 2026-02-13T02:27:51.011373+0000 (oldest deadline 2026-02-13T02:28:14.511371+0000) 2026-02-13T02:28:27.494 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:27 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:26.999+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:54.511577+0000 front 2026-02-13T02:27:54.511589+0000 (oldest deadline 2026-02-13T02:28:18.611662+0000) 2026-02-13T02:28:27.494 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:27 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:26.999+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:28:00.311981+0000 front 2026-02-13T02:28:00.311999+0000 (oldest deadline 2026-02-13T02:28:25.012036+0000) 2026-02-13T02:28:28.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:27 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:27.815+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:39.585067+0000 front 2026-02-13T02:27:39.585049+0000 (oldest deadline 2026-02-13T02:28:03.685105+0000) 2026-02-13T02:28:28.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:27 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:27.815+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.685350+0000 front 2026-02-13T02:27:43.685369+0000 (oldest deadline 2026-02-13T02:28:08.385425+0000) 2026-02-13T02:28:28.159 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:27 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:27.815+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.386146+0000 front 2026-02-13T02:27:52.386067+0000 (oldest deadline 2026-02-13T02:28:15.286116+0000) 2026-02-13T02:28:28.159 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:27 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:27.815+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:55.286247+0000 front 2026-02-13T02:27:55.286269+0000 (oldest deadline 2026-02-13T02:28:20.586295+0000) 2026-02-13T02:28:28.159 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:27 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:27.815+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:28:02.286791+0000 front 2026-02-13T02:28:02.286705+0000 (oldest deadline 2026-02-13T02:28:23.986736+0000) 2026-02-13T02:28:28.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:27 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:27.963+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:28.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:27 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:27.963+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.710229+0000 front 2026-02-13T02:27:43.710211+0000 (oldest deadline 2026-02-13T02:28:07.210302+0000) 2026-02-13T02:28:28.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:27 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:27.963+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:51.011317+0000 front 2026-02-13T02:27:51.011373+0000 (oldest deadline 2026-02-13T02:28:14.511371+0000) 2026-02-13T02:28:28.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:27 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:27.963+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:54.511577+0000 front 2026-02-13T02:27:54.511589+0000 (oldest deadline 2026-02-13T02:28:18.611662+0000) 2026-02-13T02:28:28.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:27 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:27.963+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:28:00.311981+0000 front 2026-02-13T02:28:00.311999+0000 (oldest deadline 2026-02-13T02:28:25.012036+0000) 2026-02-13T02:28:28.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:27.771+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:28.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:27.771+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:46.873217+0000 front 2026-02-13T02:27:46.873042+0000 (oldest deadline 2026-02-13T02:28:10.972948+0000) 2026-02-13T02:28:28.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:27.771+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.073281+0000 front 2026-02-13T02:27:52.073293+0000 (oldest deadline 2026-02-13T02:28:14.373359+0000) 2026-02-13T02:28:28.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:27 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:27.771+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:27:59.073720+0000 front 2026-02-13T02:27:59.073711+0000 (oldest deadline 2026-02-13T02:28:23.773773+0000) 2026-02-13T02:28:29.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:28 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:28.778+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:39.585067+0000 front 2026-02-13T02:27:39.585049+0000 (oldest deadline 2026-02-13T02:28:03.685105+0000) 2026-02-13T02:28:29.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:28 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:28.778+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.685350+0000 front 2026-02-13T02:27:43.685369+0000 (oldest deadline 2026-02-13T02:28:08.385425+0000) 2026-02-13T02:28:29.159 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:28 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:28.778+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.386146+0000 front 2026-02-13T02:27:52.386067+0000 (oldest deadline 2026-02-13T02:28:15.286116+0000) 2026-02-13T02:28:29.159 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:28 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:28.778+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:55.286247+0000 front 2026-02-13T02:27:55.286269+0000 (oldest deadline 2026-02-13T02:28:20.586295+0000) 2026-02-13T02:28:29.159 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:28 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:28.778+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:28:02.286791+0000 front 2026-02-13T02:28:02.286705+0000 (oldest deadline 2026-02-13T02:28:23.986736+0000) 2026-02-13T02:28:29.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:28 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:28.983+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:29.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:28 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:28.983+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.710229+0000 front 2026-02-13T02:27:43.710211+0000 (oldest deadline 2026-02-13T02:28:07.210302+0000) 2026-02-13T02:28:29.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:28 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:28.983+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:51.011317+0000 front 2026-02-13T02:27:51.011373+0000 (oldest deadline 2026-02-13T02:28:14.511371+0000) 2026-02-13T02:28:29.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:28 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:28.983+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:54.511577+0000 front 2026-02-13T02:27:54.511589+0000 (oldest deadline 2026-02-13T02:28:18.611662+0000) 2026-02-13T02:28:29.244 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:28 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:28.983+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:28:00.311981+0000 front 2026-02-13T02:28:00.311999+0000 (oldest deadline 2026-02-13T02:28:25.012036+0000) 2026-02-13T02:28:29.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:28 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:28.799+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:29.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:28 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:28.799+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:46.873217+0000 front 2026-02-13T02:27:46.873042+0000 (oldest deadline 2026-02-13T02:28:10.972948+0000) 2026-02-13T02:28:29.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:28 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:28.799+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.073281+0000 front 2026-02-13T02:27:52.073293+0000 (oldest deadline 2026-02-13T02:28:14.373359+0000) 2026-02-13T02:28:29.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:28 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:28.799+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:27:59.073720+0000 front 2026-02-13T02:27:59.073711+0000 (oldest deadline 2026-02-13T02:28:23.773773+0000) 2026-02-13T02:28:30.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:29 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:29.821+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:39.585067+0000 front 2026-02-13T02:27:39.585049+0000 (oldest deadline 2026-02-13T02:28:03.685105+0000) 2026-02-13T02:28:30.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:29 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:29.821+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.685350+0000 front 2026-02-13T02:27:43.685369+0000 (oldest deadline 2026-02-13T02:28:08.385425+0000) 2026-02-13T02:28:30.159 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:29 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:29.821+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.386146+0000 front 2026-02-13T02:27:52.386067+0000 (oldest deadline 2026-02-13T02:28:15.286116+0000) 2026-02-13T02:28:30.159 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:29 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:29.821+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:55.286247+0000 front 2026-02-13T02:27:55.286269+0000 (oldest deadline 2026-02-13T02:28:20.586295+0000) 2026-02-13T02:28:30.159 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:29 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:29.821+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:28:02.286791+0000 front 2026-02-13T02:28:02.286705+0000 (oldest deadline 2026-02-13T02:28:23.986736+0000) 2026-02-13T02:28:30.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:29 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:29.962+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:30.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:29 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:29.962+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.710229+0000 front 2026-02-13T02:27:43.710211+0000 (oldest deadline 2026-02-13T02:28:07.210302+0000) 2026-02-13T02:28:30.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:29 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:29.962+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:51.011317+0000 front 2026-02-13T02:27:51.011373+0000 (oldest deadline 2026-02-13T02:28:14.511371+0000) 2026-02-13T02:28:30.244 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:29 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:29.962+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:54.511577+0000 front 2026-02-13T02:27:54.511589+0000 (oldest deadline 2026-02-13T02:28:18.611662+0000) 2026-02-13T02:28:30.244 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:29 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:29.962+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:28:00.311981+0000 front 2026-02-13T02:28:00.311999+0000 (oldest deadline 2026-02-13T02:28:25.012036+0000) 2026-02-13T02:28:30.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:29 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:29.834+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:30.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:29 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:29.834+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:46.873217+0000 front 2026-02-13T02:27:46.873042+0000 (oldest deadline 2026-02-13T02:28:10.972948+0000) 2026-02-13T02:28:30.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:29 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:29.834+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.073281+0000 front 2026-02-13T02:27:52.073293+0000 (oldest deadline 2026-02-13T02:28:14.373359+0000) 2026-02-13T02:28:30.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:29 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:29.834+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:27:59.073720+0000 front 2026-02-13T02:27:59.073711+0000 (oldest deadline 2026-02-13T02:28:23.773773+0000) 2026-02-13T02:28:31.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:30 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:30.857+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:39.585067+0000 front 2026-02-13T02:27:39.585049+0000 (oldest deadline 2026-02-13T02:28:03.685105+0000) 2026-02-13T02:28:31.158 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:30 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:30.857+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.685350+0000 front 2026-02-13T02:27:43.685369+0000 (oldest deadline 2026-02-13T02:28:08.385425+0000) 2026-02-13T02:28:31.159 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:30 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:30.857+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.386146+0000 front 2026-02-13T02:27:52.386067+0000 (oldest deadline 2026-02-13T02:28:15.286116+0000) 2026-02-13T02:28:31.159 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:30 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:30.857+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:55.286247+0000 front 2026-02-13T02:27:55.286269+0000 (oldest deadline 2026-02-13T02:28:20.586295+0000) 2026-02-13T02:28:31.159 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:30 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9[55879]: 2026-02-13T02:28:30.857+0000 7f2e250fe6c0 -1 osd.9 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:28:02.286791+0000 front 2026-02-13T02:28:02.286705+0000 (oldest deadline 2026-02-13T02:28:23.986736+0000) 2026-02-13T02:28:31.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:30 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:30.917+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:31.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:30 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:30.917+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.710229+0000 front 2026-02-13T02:27:43.710211+0000 (oldest deadline 2026-02-13T02:28:07.210302+0000) 2026-02-13T02:28:31.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:30 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:30.917+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:51.011317+0000 front 2026-02-13T02:27:51.011373+0000 (oldest deadline 2026-02-13T02:28:14.511371+0000) 2026-02-13T02:28:31.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:30 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:30.917+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:54.511577+0000 front 2026-02-13T02:27:54.511589+0000 (oldest deadline 2026-02-13T02:28:18.611662+0000) 2026-02-13T02:28:31.244 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:30 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:30.917+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:28:00.311981+0000 front 2026-02-13T02:28:00.311999+0000 (oldest deadline 2026-02-13T02:28:25.012036+0000) 2026-02-13T02:28:31.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:30 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:30.818+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:31.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:30 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:30.818+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:46.873217+0000 front 2026-02-13T02:27:46.873042+0000 (oldest deadline 2026-02-13T02:28:10.972948+0000) 2026-02-13T02:28:31.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:30 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:30.818+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.073281+0000 front 2026-02-13T02:27:52.073293+0000 (oldest deadline 2026-02-13T02:28:14.373359+0000) 2026-02-13T02:28:31.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:30 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:30.818+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:27:59.073720+0000 front 2026-02-13T02:27:59.073711+0000 (oldest deadline 2026-02-13T02:28:23.773773+0000) 2026-02-13T02:28:31.584 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:31 trial027 podman[82150]: 2026-02-13 02:28:31.271072385 +0000 UTC m=+5.070368020 container died 06f527ad764d4a50da0624523a36432de00882efe95c3b4d26fa1bb71ad443b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9, org.opencontainers.image.authors=Ceph Release Team , summary=Rocky Linux Base image, version=10, license=BSD-3-Clause, org.opencontainers.image.title=rockylinux, vendor=Rocky Enterprise Software Foundation, io.buildah.version=1.33.7, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, org.opencontainers.image.version=10, name=rockylinux, OSD_FLAVOR=default, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, CEPH_REF=wip-pdonnell-testing-20260210.171210, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, ceph=True, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.licenses=BSD-3-Clause, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/) 2026-02-13T02:28:31.584 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:31 trial027 podman[82150]: 2026-02-13 02:28:31.280697036 +0000 UTC m=+5.079992661 container remove 06f527ad764d4a50da0624523a36432de00882efe95c3b4d26fa1bb71ad443b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9, CEPH_REF=wip-pdonnell-testing-20260210.171210, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, io.buildah.version=1.33.7, summary=Rocky Linux Base image, org.opencontainers.image.authors=Ceph Release Team , ceph=True, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, org.opencontainers.image.title=rockylinux, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, license=BSD-3-Clause, name=rockylinux, version=10, vendor=Rocky Enterprise Software Foundation, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, OSD_FLAVOR=default, org.opencontainers.image.licenses=BSD-3-Clause, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.version=10, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git) 2026-02-13T02:28:31.584 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:31 trial027 bash[82150]: ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9 2026-02-13T02:28:31.584 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:31 trial027 podman[82278]: 2026-02-13 02:28:31.421492515 +0000 UTC m=+0.016804633 container create 7d259fc5293496bf217bff2c3c75d32575dbed41d3858fc6893031f1488eae26 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9-deactivate, org.opencontainers.image.title=rockylinux, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, org.opencontainers.image.version=10, license=BSD-3-Clause, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, io.buildah.version=1.33.7, org.opencontainers.image.authors=Ceph Release Team , CEPH_REF=wip-pdonnell-testing-20260210.171210, name=rockylinux, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.licenses=BSD-3-Clause, vendor=Rocky Enterprise Software Foundation, summary=Rocky Linux Base image, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, ceph=True, version=10, OSD_FLAVOR=default, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git) 2026-02-13T02:28:31.584 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:31 trial027 podman[82278]: 2026-02-13 02:28:31.451457191 +0000 UTC m=+0.046769299 container init 7d259fc5293496bf217bff2c3c75d32575dbed41d3858fc6893031f1488eae26 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9-deactivate, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, summary=Rocky Linux Base image, org.opencontainers.image.version=10, name=rockylinux, license=BSD-3-Clause, org.opencontainers.image.licenses=BSD-3-Clause, vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.title=rockylinux, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, CEPH_REF=wip-pdonnell-testing-20260210.171210, OSD_FLAVOR=default, org.opencontainers.image.authors=Ceph Release Team , CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, io.buildah.version=1.33.7, ceph=True, version=10, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f) 2026-02-13T02:28:31.584 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:31 trial027 podman[82278]: 2026-02-13 02:28:31.453944419 +0000 UTC m=+0.049256547 container start 7d259fc5293496bf217bff2c3c75d32575dbed41d3858fc6893031f1488eae26 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9-deactivate, CEPH_REF=wip-pdonnell-testing-20260210.171210, name=rockylinux, version=10, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.version=10, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.licenses=BSD-3-Clause, ceph=True, OSD_FLAVOR=default, license=BSD-3-Clause, org.opencontainers.image.title=rockylinux, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, summary=Rocky Linux Base image, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, vendor=Rocky Enterprise Software Foundation, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, io.buildah.version=1.33.7) 2026-02-13T02:28:31.585 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:31 trial027 podman[82278]: 2026-02-13 02:28:31.454623379 +0000 UTC m=+0.049935517 container attach 7d259fc5293496bf217bff2c3c75d32575dbed41d3858fc6893031f1488eae26 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9-deactivate, CEPH_REF=wip-pdonnell-testing-20260210.171210, vendor=Rocky Enterprise Software Foundation, summary=Rocky Linux Base image, org.opencontainers.image.licenses=BSD-3-Clause, ceph=True, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.title=rockylinux, org.opencontainers.image.authors=Ceph Release Team , CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, name=rockylinux, license=BSD-3-Clause, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, io.buildah.version=1.33.7, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, org.opencontainers.image.version=10, version=10, OSD_FLAVOR=default, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/) 2026-02-13T02:28:31.585 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:31 trial027 podman[82278]: 2026-02-13 02:28:31.41500572 +0000 UTC m=+0.010317848 image pull b86a4a15c2dcd78db08124ee64cb48fcd681d6db032e313da19d063813b235da quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9 2026-02-13T02:28:31.585 INFO:journalctl@ceph.osd.9.trial027.stdout:Feb 13 02:28:31 trial027 podman[82278]: 2026-02-13 02:28:31.582715371 +0000 UTC m=+0.178027489 container died 7d259fc5293496bf217bff2c3c75d32575dbed41d3858fc6893031f1488eae26 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-9-deactivate, org.opencontainers.image.version=10, summary=Rocky Linux Base image, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, ceph=True, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, license=BSD-3-Clause, vendor=Rocky Enterprise Software Foundation, name=rockylinux, version=10, OSD_FLAVOR=default, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, io.buildah.version=1.33.7, org.opencontainers.image.title=rockylinux, org.opencontainers.image.authors=Ceph Release Team , CEPH_REF=wip-pdonnell-testing-20260210.171210, org.opencontainers.image.licenses=BSD-3-Clause) 2026-02-13T02:28:31.615 DEBUG:teuthology.orchestra.run.trial027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.9.service' 2026-02-13T02:28:31.691 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T02:28:31.692 INFO:tasks.cephadm.osd.9:Stopped osd.9 2026-02-13T02:28:31.692 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2026-02-13T02:28:31.692 DEBUG:teuthology.orchestra.run.trial028:> sudo systemctl stop ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.10 2026-02-13T02:28:31.993 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:31 trial028 systemd[1]: Stopping Ceph osd.10 for 707dbb29-0882-11f1-9262-d404e6e7d460... 2026-02-13T02:28:31.993 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:31 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:31.799+0000 7fec504466c0 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --osd-objectstore=bluestore (PID: 1) UID: 0 2026-02-13T02:28:31.994 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:31 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:31.799+0000 7fec504466c0 -1 osd.10 72 *** Got signal Terminated *** 2026-02-13T02:28:31.994 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:31 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:31.799+0000 7fec504466c0 -1 osd.10 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-13T02:28:31.994 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:31 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:31.889+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:31.994 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:31 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:31.889+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.710229+0000 front 2026-02-13T02:27:43.710211+0000 (oldest deadline 2026-02-13T02:28:07.210302+0000) 2026-02-13T02:28:31.994 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:31 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:31.889+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:51.011317+0000 front 2026-02-13T02:27:51.011373+0000 (oldest deadline 2026-02-13T02:28:14.511371+0000) 2026-02-13T02:28:31.994 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:31 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:31.889+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:54.511577+0000 front 2026-02-13T02:27:54.511589+0000 (oldest deadline 2026-02-13T02:28:18.611662+0000) 2026-02-13T02:28:31.994 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:31 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:31.889+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:28:00.311981+0000 front 2026-02-13T02:28:00.311999+0000 (oldest deadline 2026-02-13T02:28:25.012036+0000) 2026-02-13T02:28:32.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:31 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:31.832+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:32.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:31 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:31.832+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:46.873217+0000 front 2026-02-13T02:27:46.873042+0000 (oldest deadline 2026-02-13T02:28:10.972948+0000) 2026-02-13T02:28:32.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:31 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:31.832+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.073281+0000 front 2026-02-13T02:27:52.073293+0000 (oldest deadline 2026-02-13T02:28:14.373359+0000) 2026-02-13T02:28:32.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:31 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:31.832+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:27:59.073720+0000 front 2026-02-13T02:27:59.073711+0000 (oldest deadline 2026-02-13T02:28:23.773773+0000) 2026-02-13T02:28:33.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:32 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:32.883+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:33.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:32 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:32.883+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.710229+0000 front 2026-02-13T02:27:43.710211+0000 (oldest deadline 2026-02-13T02:28:07.210302+0000) 2026-02-13T02:28:33.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:32 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:32.883+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:51.011317+0000 front 2026-02-13T02:27:51.011373+0000 (oldest deadline 2026-02-13T02:28:14.511371+0000) 2026-02-13T02:28:33.244 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:32 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:32.883+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:54.511577+0000 front 2026-02-13T02:27:54.511589+0000 (oldest deadline 2026-02-13T02:28:18.611662+0000) 2026-02-13T02:28:33.244 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:32 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:32.883+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:28:00.311981+0000 front 2026-02-13T02:28:00.311999+0000 (oldest deadline 2026-02-13T02:28:25.012036+0000) 2026-02-13T02:28:33.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:32 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:32.863+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:33.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:32 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:32.863+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:46.873217+0000 front 2026-02-13T02:27:46.873042+0000 (oldest deadline 2026-02-13T02:28:10.972948+0000) 2026-02-13T02:28:33.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:32 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:32.863+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.073281+0000 front 2026-02-13T02:27:52.073293+0000 (oldest deadline 2026-02-13T02:28:14.373359+0000) 2026-02-13T02:28:33.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:32 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:32.863+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:27:59.073720+0000 front 2026-02-13T02:27:59.073711+0000 (oldest deadline 2026-02-13T02:28:23.773773+0000) 2026-02-13T02:28:33.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:32 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:32.863+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6812 osd.5 since back 2026-02-13T02:28:08.374365+0000 front 2026-02-13T02:28:08.374383+0000 (oldest deadline 2026-02-13T02:28:32.474566+0000) 2026-02-13T02:28:34.144 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:33 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:33.857+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:34.144 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:33 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:33.857+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.710229+0000 front 2026-02-13T02:27:43.710211+0000 (oldest deadline 2026-02-13T02:28:07.210302+0000) 2026-02-13T02:28:34.145 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:33 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:33.857+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:51.011317+0000 front 2026-02-13T02:27:51.011373+0000 (oldest deadline 2026-02-13T02:28:14.511371+0000) 2026-02-13T02:28:34.145 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:33 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:33.857+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:54.511577+0000 front 2026-02-13T02:27:54.511589+0000 (oldest deadline 2026-02-13T02:28:18.611662+0000) 2026-02-13T02:28:34.145 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:33 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:33.857+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:28:00.311981+0000 front 2026-02-13T02:28:00.311999+0000 (oldest deadline 2026-02-13T02:28:25.012036+0000) 2026-02-13T02:28:34.145 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:33 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:33.857+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.107:6812 osd.5 since back 2026-02-13T02:28:08.512462+0000 front 2026-02-13T02:28:08.512417+0000 (oldest deadline 2026-02-13T02:28:33.812436+0000) 2026-02-13T02:28:34.250 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:33 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:33.907+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:34.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:33 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:33.907+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:46.873217+0000 front 2026-02-13T02:27:46.873042+0000 (oldest deadline 2026-02-13T02:28:10.972948+0000) 2026-02-13T02:28:34.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:33 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:33.907+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.073281+0000 front 2026-02-13T02:27:52.073293+0000 (oldest deadline 2026-02-13T02:28:14.373359+0000) 2026-02-13T02:28:34.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:33 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:33.907+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:27:59.073720+0000 front 2026-02-13T02:27:59.073711+0000 (oldest deadline 2026-02-13T02:28:23.773773+0000) 2026-02-13T02:28:34.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:33 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:33.907+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6812 osd.5 since back 2026-02-13T02:28:08.374365+0000 front 2026-02-13T02:28:08.374383+0000 (oldest deadline 2026-02-13T02:28:32.474566+0000) 2026-02-13T02:28:35.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:34 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:34.857+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:35.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:34 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:34.857+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.710229+0000 front 2026-02-13T02:27:43.710211+0000 (oldest deadline 2026-02-13T02:28:07.210302+0000) 2026-02-13T02:28:35.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:34 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:34.857+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:51.011317+0000 front 2026-02-13T02:27:51.011373+0000 (oldest deadline 2026-02-13T02:28:14.511371+0000) 2026-02-13T02:28:35.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:34 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:34.857+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:54.511577+0000 front 2026-02-13T02:27:54.511589+0000 (oldest deadline 2026-02-13T02:28:18.611662+0000) 2026-02-13T02:28:35.244 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:34 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:34.857+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:28:00.311981+0000 front 2026-02-13T02:28:00.311999+0000 (oldest deadline 2026-02-13T02:28:25.012036+0000) 2026-02-13T02:28:35.244 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:34 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:34.857+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.107:6812 osd.5 since back 2026-02-13T02:28:08.512462+0000 front 2026-02-13T02:28:08.512417+0000 (oldest deadline 2026-02-13T02:28:33.812436+0000) 2026-02-13T02:28:35.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:34 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:34.900+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:35.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:34 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:34.900+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:46.873217+0000 front 2026-02-13T02:27:46.873042+0000 (oldest deadline 2026-02-13T02:28:10.972948+0000) 2026-02-13T02:28:35.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:34 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:34.900+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.073281+0000 front 2026-02-13T02:27:52.073293+0000 (oldest deadline 2026-02-13T02:28:14.373359+0000) 2026-02-13T02:28:35.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:34 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:34.900+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:27:59.073720+0000 front 2026-02-13T02:27:59.073711+0000 (oldest deadline 2026-02-13T02:28:23.773773+0000) 2026-02-13T02:28:35.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:34 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:34.900+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6812 osd.5 since back 2026-02-13T02:28:08.374365+0000 front 2026-02-13T02:28:08.374383+0000 (oldest deadline 2026-02-13T02:28:32.474566+0000) 2026-02-13T02:28:36.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:35 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:35.818+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:36.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:35 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:35.818+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.710229+0000 front 2026-02-13T02:27:43.710211+0000 (oldest deadline 2026-02-13T02:28:07.210302+0000) 2026-02-13T02:28:36.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:35 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:35.818+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:51.011317+0000 front 2026-02-13T02:27:51.011373+0000 (oldest deadline 2026-02-13T02:28:14.511371+0000) 2026-02-13T02:28:36.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:35 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:35.818+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:54.511577+0000 front 2026-02-13T02:27:54.511589+0000 (oldest deadline 2026-02-13T02:28:18.611662+0000) 2026-02-13T02:28:36.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:35 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:35.818+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:28:00.311981+0000 front 2026-02-13T02:28:00.311999+0000 (oldest deadline 2026-02-13T02:28:25.012036+0000) 2026-02-13T02:28:36.243 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:35 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:35.818+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.107:6812 osd.5 since back 2026-02-13T02:28:08.512462+0000 front 2026-02-13T02:28:08.512417+0000 (oldest deadline 2026-02-13T02:28:33.812436+0000) 2026-02-13T02:28:36.250 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:35 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:35.875+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:36.250 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:35 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:35.875+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:46.873217+0000 front 2026-02-13T02:27:46.873042+0000 (oldest deadline 2026-02-13T02:28:10.972948+0000) 2026-02-13T02:28:36.250 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:35 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:35.875+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.073281+0000 front 2026-02-13T02:27:52.073293+0000 (oldest deadline 2026-02-13T02:28:14.373359+0000) 2026-02-13T02:28:36.250 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:35 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:35.875+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:27:59.073720+0000 front 2026-02-13T02:27:59.073711+0000 (oldest deadline 2026-02-13T02:28:23.773773+0000) 2026-02-13T02:28:36.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:35 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:35.875+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6812 osd.5 since back 2026-02-13T02:28:08.374365+0000 front 2026-02-13T02:28:08.374383+0000 (oldest deadline 2026-02-13T02:28:32.474566+0000) 2026-02-13T02:28:37.088 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:36 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:36.797+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:40.810224+0000 front 2026-02-13T02:27:40.810058+0000 (oldest deadline 2026-02-13T02:28:03.710079+0000) 2026-02-13T02:28:37.088 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:36 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:36.797+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:43.710229+0000 front 2026-02-13T02:27:43.710211+0000 (oldest deadline 2026-02-13T02:28:07.210302+0000) 2026-02-13T02:28:37.088 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:36 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:36.797+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:51.011317+0000 front 2026-02-13T02:27:51.011373+0000 (oldest deadline 2026-02-13T02:28:14.511371+0000) 2026-02-13T02:28:37.089 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:36 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:36.797+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6814 osd.3 since back 2026-02-13T02:27:54.511577+0000 front 2026-02-13T02:27:54.511589+0000 (oldest deadline 2026-02-13T02:28:18.611662+0000) 2026-02-13T02:28:37.089 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:36 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:36.797+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:28:00.311981+0000 front 2026-02-13T02:28:00.311999+0000 (oldest deadline 2026-02-13T02:28:25.012036+0000) 2026-02-13T02:28:37.089 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:36 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:36.797+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.107:6812 osd.5 since back 2026-02-13T02:28:08.512462+0000 front 2026-02-13T02:28:08.512417+0000 (oldest deadline 2026-02-13T02:28:33.812436+0000) 2026-02-13T02:28:37.089 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:36 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10[50310]: 2026-02-13T02:28:36.797+0000 7fec48b036c0 -1 osd.10 72 heartbeat_check: no reply from 10.20.193.27:6822 osd.6 since back 2026-02-13T02:28:13.812672+0000 front 2026-02-13T02:28:13.812760+0000 (oldest deadline 2026-02-13T02:28:36.712668+0000) 2026-02-13T02:28:37.089 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:36 trial028 podman[58389]: 2026-02-13 02:28:36.85933805 +0000 UTC m=+5.071711777 container died 9317751edfbb1192a11e00540f3f7028a6bc154b4e179b0f01acd2598332590e (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10, org.opencontainers.image.version=10, OSD_FLAVOR=default, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, license=BSD-3-Clause, org.opencontainers.image.licenses=BSD-3-Clause, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, org.opencontainers.image.authors=Ceph Release Team , version=10, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, name=rockylinux, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, summary=Rocky Linux Base image, vendor=Rocky Enterprise Software Foundation, CEPH_REF=wip-pdonnell-testing-20260210.171210, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, ceph=True, io.buildah.version=1.33.7, org.opencontainers.image.title=rockylinux) 2026-02-13T02:28:37.089 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:36 trial028 podman[58389]: 2026-02-13 02:28:36.867534788 +0000 UTC m=+5.079908505 container remove 9317751edfbb1192a11e00540f3f7028a6bc154b4e179b0f01acd2598332590e (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.version=10, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, io.buildah.version=1.33.7, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.title=rockylinux, vendor=Rocky Enterprise Software Foundation, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, version=10, license=BSD-3-Clause, OSD_FLAVOR=default, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, summary=Rocky Linux Base image, ceph=True, name=rockylinux, org.opencontainers.image.licenses=BSD-3-Clause, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, CEPH_REF=wip-pdonnell-testing-20260210.171210, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation) 2026-02-13T02:28:37.089 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:36 trial028 bash[58389]: ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10 2026-02-13T02:28:37.089 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:36 trial028 podman[58516]: 2026-02-13 02:28:36.99872228 +0000 UTC m=+0.016204305 container create 9dcbd4604451f5618217d09f1bedbecbf5d18ef6dc0b0736e1641c42db8d859e (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10-deactivate, license=BSD-3-Clause, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, version=10, org.opencontainers.image.title=rockylinux, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, io.buildah.version=1.33.7, name=rockylinux, org.opencontainers.image.documentation=https://docs.ceph.com/, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, vendor=Rocky Enterprise Software Foundation, CEPH_REF=wip-pdonnell-testing-20260210.171210, org.opencontainers.image.version=10, OSD_FLAVOR=default, org.opencontainers.image.authors=Ceph Release Team , summary=Rocky Linux Base image, ceph=True, org.opencontainers.image.licenses=BSD-3-Clause, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation) 2026-02-13T02:28:37.089 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:37 trial028 podman[58516]: 2026-02-13 02:28:37.025331914 +0000 UTC m=+0.042813869 container init 9dcbd4604451f5618217d09f1bedbecbf5d18ef6dc0b0736e1641c42db8d859e (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10-deactivate, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, org.opencontainers.image.licenses=BSD-3-Clause, license=BSD-3-Clause, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, version=10, name=rockylinux, OSD_FLAVOR=default, summary=Rocky Linux Base image, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, CEPH_REF=wip-pdonnell-testing-20260210.171210, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, io.buildah.version=1.33.7, org.opencontainers.image.version=10, org.opencontainers.image.title=rockylinux, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, vendor=Rocky Enterprise Software Foundation, ceph=True, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/) 2026-02-13T02:28:37.089 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:37 trial028 podman[58516]: 2026-02-13 02:28:37.027987253 +0000 UTC m=+0.045469208 container start 9dcbd4604451f5618217d09f1bedbecbf5d18ef6dc0b0736e1641c42db8d859e (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10-deactivate, summary=Rocky Linux Base image, vendor=Rocky Enterprise Software Foundation, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, CEPH_REF=wip-pdonnell-testing-20260210.171210, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, version=10, name=rockylinux, OSD_FLAVOR=default, org.opencontainers.image.authors=Ceph Release Team , GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, license=BSD-3-Clause, io.buildah.version=1.33.7, ceph=True, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, org.opencontainers.image.licenses=BSD-3-Clause, org.opencontainers.image.version=10, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.title=rockylinux, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f) 2026-02-13T02:28:37.090 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:37 trial028 podman[58516]: 2026-02-13 02:28:37.028694957 +0000 UTC m=+0.046176912 container attach 9dcbd4604451f5618217d09f1bedbecbf5d18ef6dc0b0736e1641c42db8d859e (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-10-deactivate, CEPH_REF=wip-pdonnell-testing-20260210.171210, license=BSD-3-Clause, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, org.opencontainers.image.version=10, org.opencontainers.image.licenses=BSD-3-Clause, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, vendor=Rocky Enterprise Software Foundation, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, version=10, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.title=rockylinux, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, io.buildah.version=1.33.7, name=rockylinux, OSD_FLAVOR=default, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, summary=Rocky Linux Base image, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, ceph=True) 2026-02-13T02:28:37.090 INFO:journalctl@ceph.osd.10.trial028.stdout:Feb 13 02:28:37 trial028 podman[58516]: 2026-02-13 02:28:36.992328299 +0000 UTC m=+0.009810264 image pull b86a4a15c2dcd78db08124ee64cb48fcd681d6db032e313da19d063813b235da quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9 2026-02-13T02:28:37.199 DEBUG:teuthology.orchestra.run.trial028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.10.service' 2026-02-13T02:28:37.235 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T02:28:37.235 INFO:tasks.cephadm.osd.10:Stopped osd.10 2026-02-13T02:28:37.235 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2026-02-13T02:28:37.235 DEBUG:teuthology.orchestra.run.trial107:> sudo systemctl stop ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.11 2026-02-13T02:28:37.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:36 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:36.865+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:37.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:36 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:36.865+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:46.873217+0000 front 2026-02-13T02:27:46.873042+0000 (oldest deadline 2026-02-13T02:28:10.972948+0000) 2026-02-13T02:28:37.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:36 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:36.865+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.073281+0000 front 2026-02-13T02:27:52.073293+0000 (oldest deadline 2026-02-13T02:28:14.373359+0000) 2026-02-13T02:28:37.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:36 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:36.865+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:27:59.073720+0000 front 2026-02-13T02:27:59.073711+0000 (oldest deadline 2026-02-13T02:28:23.773773+0000) 2026-02-13T02:28:37.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:36 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:36.865+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6812 osd.5 since back 2026-02-13T02:28:08.374365+0000 front 2026-02-13T02:28:08.374383+0000 (oldest deadline 2026-02-13T02:28:32.474566+0000) 2026-02-13T02:28:37.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:36 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:36.865+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6822 osd.6 since back 2026-02-13T02:28:12.474882+0000 front 2026-02-13T02:28:12.474925+0000 (oldest deadline 2026-02-13T02:28:36.574820+0000) 2026-02-13T02:28:37.751 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:37 trial107 systemd[1]: Stopping Ceph osd.11 for 707dbb29-0882-11f1-9262-d404e6e7d460... 2026-02-13T02:28:37.751 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:37 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:37.338+0000 7fc6a334e6c0 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --osd-objectstore=bluestore (PID: 1) UID: 0 2026-02-13T02:28:37.751 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:37 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:37.338+0000 7fc6a334e6c0 -1 osd.11 72 *** Got signal Terminated *** 2026-02-13T02:28:37.751 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:37 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:37.338+0000 7fc6a334e6c0 -1 osd.11 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-13T02:28:38.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:37 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:37.864+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:38.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:37 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:37.864+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:46.873217+0000 front 2026-02-13T02:27:46.873042+0000 (oldest deadline 2026-02-13T02:28:10.972948+0000) 2026-02-13T02:28:38.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:37 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:37.864+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.073281+0000 front 2026-02-13T02:27:52.073293+0000 (oldest deadline 2026-02-13T02:28:14.373359+0000) 2026-02-13T02:28:38.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:37 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:37.864+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:27:59.073720+0000 front 2026-02-13T02:27:59.073711+0000 (oldest deadline 2026-02-13T02:28:23.773773+0000) 2026-02-13T02:28:38.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:37 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:37.864+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6812 osd.5 since back 2026-02-13T02:28:08.374365+0000 front 2026-02-13T02:28:08.374383+0000 (oldest deadline 2026-02-13T02:28:32.474566+0000) 2026-02-13T02:28:38.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:37 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:37.864+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6822 osd.6 since back 2026-02-13T02:28:12.474882+0000 front 2026-02-13T02:28:12.474925+0000 (oldest deadline 2026-02-13T02:28:36.574820+0000) 2026-02-13T02:28:39.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:38 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:38.856+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:39.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:38 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:38.856+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:46.873217+0000 front 2026-02-13T02:27:46.873042+0000 (oldest deadline 2026-02-13T02:28:10.972948+0000) 2026-02-13T02:28:39.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:38 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:38.856+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.073281+0000 front 2026-02-13T02:27:52.073293+0000 (oldest deadline 2026-02-13T02:28:14.373359+0000) 2026-02-13T02:28:39.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:38 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:38.856+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:27:59.073720+0000 front 2026-02-13T02:27:59.073711+0000 (oldest deadline 2026-02-13T02:28:23.773773+0000) 2026-02-13T02:28:39.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:38 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:38.856+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6812 osd.5 since back 2026-02-13T02:28:08.374365+0000 front 2026-02-13T02:28:08.374383+0000 (oldest deadline 2026-02-13T02:28:32.474566+0000) 2026-02-13T02:28:39.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:38 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:38.856+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6822 osd.6 since back 2026-02-13T02:28:12.474882+0000 front 2026-02-13T02:28:12.474925+0000 (oldest deadline 2026-02-13T02:28:36.574820+0000) 2026-02-13T02:28:40.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:39 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:39.823+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:40.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:39 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:39.823+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:46.873217+0000 front 2026-02-13T02:27:46.873042+0000 (oldest deadline 2026-02-13T02:28:10.972948+0000) 2026-02-13T02:28:40.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:39 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:39.823+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.073281+0000 front 2026-02-13T02:27:52.073293+0000 (oldest deadline 2026-02-13T02:28:14.373359+0000) 2026-02-13T02:28:40.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:39 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:39.823+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:27:59.073720+0000 front 2026-02-13T02:27:59.073711+0000 (oldest deadline 2026-02-13T02:28:23.773773+0000) 2026-02-13T02:28:40.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:39 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:39.823+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6812 osd.5 since back 2026-02-13T02:28:08.374365+0000 front 2026-02-13T02:28:08.374383+0000 (oldest deadline 2026-02-13T02:28:32.474566+0000) 2026-02-13T02:28:40.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:39 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:39.823+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6822 osd.6 since back 2026-02-13T02:28:12.474882+0000 front 2026-02-13T02:28:12.474925+0000 (oldest deadline 2026-02-13T02:28:36.574820+0000) 2026-02-13T02:28:41.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:40 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:40.784+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:41.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:40 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:40.784+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:46.873217+0000 front 2026-02-13T02:27:46.873042+0000 (oldest deadline 2026-02-13T02:28:10.972948+0000) 2026-02-13T02:28:41.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:40 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:40.784+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.073281+0000 front 2026-02-13T02:27:52.073293+0000 (oldest deadline 2026-02-13T02:28:14.373359+0000) 2026-02-13T02:28:41.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:40 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:40.784+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:27:59.073720+0000 front 2026-02-13T02:27:59.073711+0000 (oldest deadline 2026-02-13T02:28:23.773773+0000) 2026-02-13T02:28:41.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:40 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:40.784+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6812 osd.5 since back 2026-02-13T02:28:08.374365+0000 front 2026-02-13T02:28:08.374383+0000 (oldest deadline 2026-02-13T02:28:32.474566+0000) 2026-02-13T02:28:41.251 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:40 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:40.784+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6822 osd.6 since back 2026-02-13T02:28:12.474882+0000 front 2026-02-13T02:28:12.474925+0000 (oldest deadline 2026-02-13T02:28:36.574820+0000) 2026-02-13T02:28:41.252 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:40 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:40.784+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6820 osd.7 since back 2026-02-13T02:28:17.675271+0000 front 2026-02-13T02:28:17.675287+0000 (oldest deadline 2026-02-13T02:28:40.575230+0000) 2026-02-13T02:28:42.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:41 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:41.751+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6806 osd.0 since back 2026-02-13T02:27:35.672376+0000 front 2026-02-13T02:27:35.672413+0000 (oldest deadline 2026-02-13T02:28:01.572419+0000) 2026-02-13T02:28:42.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:41 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:41.751+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6804 osd.1 since back 2026-02-13T02:27:46.873217+0000 front 2026-02-13T02:27:46.873042+0000 (oldest deadline 2026-02-13T02:28:10.972948+0000) 2026-02-13T02:28:42.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:41 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:41.751+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6804 osd.2 since back 2026-02-13T02:27:52.073281+0000 front 2026-02-13T02:27:52.073293+0000 (oldest deadline 2026-02-13T02:28:14.373359+0000) 2026-02-13T02:28:42.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:41 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:41.751+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6812 osd.4 since back 2026-02-13T02:27:59.073720+0000 front 2026-02-13T02:27:59.073711+0000 (oldest deadline 2026-02-13T02:28:23.773773+0000) 2026-02-13T02:28:42.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:41 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:41.751+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.107:6812 osd.5 since back 2026-02-13T02:28:08.374365+0000 front 2026-02-13T02:28:08.374383+0000 (oldest deadline 2026-02-13T02:28:32.474566+0000) 2026-02-13T02:28:42.001 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:41 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:41.751+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.27:6822 osd.6 since back 2026-02-13T02:28:12.474882+0000 front 2026-02-13T02:28:12.474925+0000 (oldest deadline 2026-02-13T02:28:36.574820+0000) 2026-02-13T02:28:42.002 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:41 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11[51402]: 2026-02-13T02:28:41.751+0000 7fc69fa136c0 -1 osd.11 72 heartbeat_check: no reply from 10.20.193.28:6820 osd.7 since back 2026-02-13T02:28:17.675271+0000 front 2026-02-13T02:28:17.675287+0000 (oldest deadline 2026-02-13T02:28:40.575230+0000) 2026-02-13T02:28:42.710 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:42 trial107 podman[60962]: 2026-02-13 02:28:42.408168085 +0000 UTC m=+5.082138042 container died d7a34a08d014e471749638e03604c6e1771bfb530a75037e6e22122bb2fdccc2 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11, org.opencontainers.image.title=rockylinux, org.opencontainers.image.authors=Ceph Release Team , version=10, license=BSD-3-Clause, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, OSD_FLAVOR=default, org.opencontainers.image.version=10, summary=Rocky Linux Base image, CEPH_REF=wip-pdonnell-testing-20260210.171210, io.buildah.version=1.33.7, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, vendor=Rocky Enterprise Software Foundation, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, ceph=True, name=rockylinux, org.opencontainers.image.licenses=BSD-3-Clause) 2026-02-13T02:28:42.710 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:42 trial107 podman[60962]: 2026-02-13 02:28:42.416605525 +0000 UTC m=+5.090575472 container remove d7a34a08d014e471749638e03604c6e1771bfb530a75037e6e22122bb2fdccc2 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11, license=BSD-3-Clause, vendor=Rocky Enterprise Software Foundation, CEPH_REF=wip-pdonnell-testing-20260210.171210, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, ceph=True, io.buildah.version=1.33.7, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.documentation=https://docs.ceph.com/, name=rockylinux, org.opencontainers.image.title=rockylinux, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.version=10, version=10, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, summary=Rocky Linux Base image, org.opencontainers.image.authors=Ceph Release Team , OSD_FLAVOR=default, org.opencontainers.image.licenses=BSD-3-Clause) 2026-02-13T02:28:42.710 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:42 trial107 bash[60962]: ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11 2026-02-13T02:28:42.710 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:42 trial107 podman[61092]: 2026-02-13 02:28:42.549210309 +0000 UTC m=+0.016142592 container create ce40786d70ac881af44c598a22a730183d21a429ce7ef32f95ff08cc2ac308f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11-deactivate, summary=Rocky Linux Base image, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, io.buildah.version=1.33.7, vendor=Rocky Enterprise Software Foundation, ceph=True, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, license=BSD-3-Clause, org.opencontainers.image.licenses=BSD-3-Clause, OSD_FLAVOR=default, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, name=rockylinux, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.authors=Ceph Release Team , CEPH_REF=wip-pdonnell-testing-20260210.171210, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.title=rockylinux, version=10, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.version=10) 2026-02-13T02:28:42.710 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:42 trial107 podman[61092]: 2026-02-13 02:28:42.578204469 +0000 UTC m=+0.045136692 container init ce40786d70ac881af44c598a22a730183d21a429ce7ef32f95ff08cc2ac308f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11-deactivate, version=10, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, OSD_FLAVOR=default, org.opencontainers.image.authors=Ceph Release Team , summary=Rocky Linux Base image, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, io.buildah.version=1.33.7, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.licenses=BSD-3-Clause, ceph=True, org.opencontainers.image.title=rockylinux, vendor=Rocky Enterprise Software Foundation, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, license=BSD-3-Clause, name=rockylinux, org.opencontainers.image.version=10, CEPH_REF=wip-pdonnell-testing-20260210.171210, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/) 2026-02-13T02:28:42.710 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:42 trial107 podman[61092]: 2026-02-13 02:28:42.580752892 +0000 UTC m=+0.047685126 container start ce40786d70ac881af44c598a22a730183d21a429ce7ef32f95ff08cc2ac308f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11-deactivate, OSD_FLAVOR=default, CEPH_REF=wip-pdonnell-testing-20260210.171210, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, org.opencontainers.image.documentation=https://docs.ceph.com/, ceph=True, version=10, license=BSD-3-Clause, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.title=rockylinux, name=rockylinux, org.opencontainers.image.version=10, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, org.opencontainers.image.licenses=BSD-3-Clause, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, summary=Rocky Linux Base image, io.buildah.version=1.33.7, vendor=Rocky Enterprise Software Foundation, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation) 2026-02-13T02:28:42.710 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:42 trial107 podman[61092]: 2026-02-13 02:28:42.581299908 +0000 UTC m=+0.048232131 container attach ce40786d70ac881af44c598a22a730183d21a429ce7ef32f95ff08cc2ac308f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11-deactivate, org.opencontainers.image.authors=Ceph Release Team , summary=Rocky Linux Base image, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.version=10, org.opencontainers.image.title=rockylinux, version=10, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, name=rockylinux, CEPH_REF=wip-pdonnell-testing-20260210.171210, org.opencontainers.image.documentation=https://docs.ceph.com/, OSD_FLAVOR=default, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10, io.buildah.version=1.33.7, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, ceph=True, license=BSD-3-Clause, org.opencontainers.image.licenses=BSD-3-Clause, vendor=Rocky Enterprise Software Foundation) 2026-02-13T02:28:42.710 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:42 trial107 podman[61092]: 2026-02-13 02:28:42.54269084 +0000 UTC m=+0.009623064 image pull b86a4a15c2dcd78db08124ee64cb48fcd681d6db032e313da19d063813b235da quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9 2026-02-13T02:28:42.710 INFO:journalctl@ceph.osd.11.trial107.stdout:Feb 13 02:28:42 trial107 podman[61092]: 2026-02-13 02:28:42.712609146 +0000 UTC m=+0.179541369 container died ce40786d70ac881af44c598a22a730183d21a429ce7ef32f95ff08cc2ac308f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:63f911038232e331f6e269b8b72d48700ca549f85e7c5b5176a58489c268cee9, name=ceph-707dbb29-0882-11f1-9262-d404e6e7d460-osd-11-deactivate, ceph=True, name=rockylinux, org.opencontainers.image.licenses=BSD-3-Clause, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.authors=Ceph Release Team , summary=Rocky Linux Base image, io.buildah.version=1.33.7, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=docker.io/rockylinux/rockylinux:10, OSD_FLAVOR=default, vendor=Rocky Enterprise Software Foundation, license=BSD-3-Clause, org.opencontainers.image.vendor=Rocky Enterprise Software Foundation, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_REF=wip-pdonnell-testing-20260210.171210, org.opencontainers.image.title=rockylinux, version=10, org.opencontainers.image.version=10, CEPH_SHA1=8f8d38294e4db8750b4b1649f62ce37caa63080f, org.opencontainers.image.source=https://git.resf.org/sig_core/rocky-kiwi-descriptions/src/branch/r10) 2026-02-13T02:28:42.743 DEBUG:teuthology.orchestra.run.trial107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@osd.11.service' 2026-02-13T02:28:42.818 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T02:28:42.819 INFO:tasks.cephadm.osd.11:Stopped osd.11 2026-02-13T02:28:42.819 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2026-02-13T02:28:42.819 DEBUG:teuthology.orchestra.run.trial027:> sudo systemctl stop ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.a 2026-02-13T02:28:43.158 INFO:journalctl@ceph.mds.a.trial027.stdout:Feb 13 02:28:42 trial027 systemd[1]: Stopping Ceph mds.a for 707dbb29-0882-11f1-9262-d404e6e7d460... 2026-02-13T02:28:43.158 INFO:journalctl@ceph.mds.a.trial027.stdout:Feb 13 02:28:42 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-a[61693]: 2026-02-13T02:28:42.914+0000 7f0f16fc56c0 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2026-02-13T02:28:43.159 INFO:journalctl@ceph.mds.a.trial027.stdout:Feb 13 02:28:42 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-a[61693]: 2026-02-13T02:28:42.914+0000 7f0f16fc56c0 -1 mds.a *** got signal Terminated *** 2026-02-13T02:28:43.971 DEBUG:teuthology.orchestra.run.trial027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.a.service' 2026-02-13T02:28:44.006 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T02:28:44.006 INFO:tasks.cephadm.mds.a:Stopped mds.a 2026-02-13T02:28:44.007 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2026-02-13T02:28:44.007 DEBUG:teuthology.orchestra.run.trial027:> sudo systemctl stop ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.d 2026-02-13T02:28:44.137 INFO:journalctl@ceph.mds.d.trial027.stdout:Feb 13 02:28:44 trial027 systemd[1]: Stopping Ceph mds.d for 707dbb29-0882-11f1-9262-d404e6e7d460... 2026-02-13T02:28:44.408 INFO:journalctl@ceph.mds.d.trial027.stdout:Feb 13 02:28:44 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-d[60835]: 2026-02-13T02:28:44.135+0000 7f9178a246c0 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2026-02-13T02:28:44.408 INFO:journalctl@ceph.mds.d.trial027.stdout:Feb 13 02:28:44 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-d[60835]: 2026-02-13T02:28:44.135+0000 7f9178a246c0 -1 mds.d *** got signal Terminated *** 2026-02-13T02:28:45.180 DEBUG:teuthology.orchestra.run.trial027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.d.service' 2026-02-13T02:28:45.215 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T02:28:45.215 INFO:tasks.cephadm.mds.d:Stopped mds.d 2026-02-13T02:28:45.215 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2026-02-13T02:28:45.215 DEBUG:teuthology.orchestra.run.trial027:> sudo systemctl stop ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.g 2026-02-13T02:28:45.408 INFO:journalctl@ceph.mds.g.trial027.stdout:Feb 13 02:28:45 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-g[61368]: 2026-02-13T02:28:45.344+0000 7f7c3f3df6c0 -1 mds.g *** got signal Terminated *** 2026-02-13T02:28:46.391 DEBUG:teuthology.orchestra.run.trial027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.g.service' 2026-02-13T02:28:46.424 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T02:28:46.424 INFO:tasks.cephadm.mds.g:Stopped mds.g 2026-02-13T02:28:46.425 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2026-02-13T02:28:46.425 DEBUG:teuthology.orchestra.run.trial027:> sudo systemctl stop ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.j 2026-02-13T02:28:46.658 INFO:journalctl@ceph.mds.j.trial027.stdout:Feb 13 02:28:46 trial027 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-j[60469]: 2026-02-13T02:28:46.553+0000 7f6278ffd6c0 -1 mds.j *** got signal Terminated *** 2026-02-13T02:28:47.585 DEBUG:teuthology.orchestra.run.trial027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.j.service' 2026-02-13T02:28:47.617 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T02:28:47.618 INFO:tasks.cephadm.mds.j:Stopped mds.j 2026-02-13T02:28:47.618 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2026-02-13T02:28:47.618 DEBUG:teuthology.orchestra.run.trial028:> sudo systemctl stop ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.b 2026-02-13T02:28:47.993 INFO:journalctl@ceph.mds.b.trial028.stdout:Feb 13 02:28:47 trial028 systemd[1]: Stopping Ceph mds.b for 707dbb29-0882-11f1-9262-d404e6e7d460... 2026-02-13T02:28:47.993 INFO:journalctl@ceph.mds.b.trial028.stdout:Feb 13 02:28:47 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-b[54870]: 2026-02-13T02:28:47.722+0000 7f565e2696c0 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2026-02-13T02:28:47.993 INFO:journalctl@ceph.mds.b.trial028.stdout:Feb 13 02:28:47 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-b[54870]: 2026-02-13T02:28:47.722+0000 7f565e2696c0 -1 mds.b *** got signal Terminated *** 2026-02-13T02:28:53.294 DEBUG:teuthology.orchestra.run.trial028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.b.service' 2026-02-13T02:28:53.328 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T02:28:53.329 INFO:tasks.cephadm.mds.b:Stopped mds.b 2026-02-13T02:28:53.329 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2026-02-13T02:28:53.329 DEBUG:teuthology.orchestra.run.trial028:> sudo systemctl stop ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.e 2026-02-13T02:28:53.467 INFO:journalctl@ceph.mds.e.trial028.stdout:Feb 13 02:28:53 trial028 systemd[1]: Stopping Ceph mds.e for 707dbb29-0882-11f1-9262-d404e6e7d460... 2026-02-13T02:28:53.743 INFO:journalctl@ceph.mds.e.trial028.stdout:Feb 13 02:28:53 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-e[54220]: 2026-02-13T02:28:53.469+0000 7f3e437656c0 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2026-02-13T02:28:53.743 INFO:journalctl@ceph.mds.e.trial028.stdout:Feb 13 02:28:53 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-e[54220]: 2026-02-13T02:28:53.469+0000 7f3e437656c0 -1 mds.e *** got signal Terminated *** 2026-02-13T02:28:54.508 DEBUG:teuthology.orchestra.run.trial028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.e.service' 2026-02-13T02:28:54.542 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T02:28:54.542 INFO:tasks.cephadm.mds.e:Stopped mds.e 2026-02-13T02:28:54.542 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2026-02-13T02:28:54.542 DEBUG:teuthology.orchestra.run.trial028:> sudo systemctl stop ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.h 2026-02-13T02:28:54.681 INFO:journalctl@ceph.mds.h.trial028.stdout:Feb 13 02:28:54 trial028 systemd[1]: Stopping Ceph mds.h for 707dbb29-0882-11f1-9262-d404e6e7d460... 2026-02-13T02:28:54.993 INFO:journalctl@ceph.mds.h.trial028.stdout:Feb 13 02:28:54 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-h[54547]: 2026-02-13T02:28:54.683+0000 7f53fcc446c0 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2026-02-13T02:28:54.993 INFO:journalctl@ceph.mds.h.trial028.stdout:Feb 13 02:28:54 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-h[54547]: 2026-02-13T02:28:54.683+0000 7f53fcc446c0 -1 mds.h *** got signal Terminated *** 2026-02-13T02:28:55.795 DEBUG:teuthology.orchestra.run.trial028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.h.service' 2026-02-13T02:28:55.828 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T02:28:55.828 INFO:tasks.cephadm.mds.h:Stopped mds.h 2026-02-13T02:28:55.829 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2026-02-13T02:28:55.829 DEBUG:teuthology.orchestra.run.trial028:> sudo systemctl stop ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.k 2026-02-13T02:28:55.993 INFO:journalctl@ceph.mds.k.trial028.stdout:Feb 13 02:28:55 trial028 systemd[1]: Stopping Ceph mds.k for 707dbb29-0882-11f1-9262-d404e6e7d460... 2026-02-13T02:28:55.993 INFO:journalctl@ceph.mds.k.trial028.stdout:Feb 13 02:28:55 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-k[53896]: 2026-02-13T02:28:55.963+0000 7fb50a6cf6c0 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2026-02-13T02:28:55.993 INFO:journalctl@ceph.mds.k.trial028.stdout:Feb 13 02:28:55 trial028 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-k[53896]: 2026-02-13T02:28:55.963+0000 7fb50a6cf6c0 -1 mds.k *** got signal Terminated *** 2026-02-13T02:28:56.995 DEBUG:teuthology.orchestra.run.trial028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.k.service' 2026-02-13T02:28:57.030 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T02:28:57.030 INFO:tasks.cephadm.mds.k:Stopped mds.k 2026-02-13T02:28:57.030 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2026-02-13T02:28:57.030 DEBUG:teuthology.orchestra.run.trial107:> sudo systemctl stop ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.c 2026-02-13T02:28:57.414 INFO:journalctl@ceph.mds.c.trial107.stdout:Feb 13 02:28:57 trial107 systemd[1]: Stopping Ceph mds.c for 707dbb29-0882-11f1-9262-d404e6e7d460... 2026-02-13T02:28:57.414 INFO:journalctl@ceph.mds.c.trial107.stdout:Feb 13 02:28:57 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-c[55965]: 2026-02-13T02:28:57.128+0000 7f145f6506c0 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2026-02-13T02:28:57.414 INFO:journalctl@ceph.mds.c.trial107.stdout:Feb 13 02:28:57 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-c[55965]: 2026-02-13T02:28:57.128+0000 7f145f6506c0 -1 mds.c *** got signal Terminated *** 2026-02-13T02:28:58.159 DEBUG:teuthology.orchestra.run.trial107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.c.service' 2026-02-13T02:28:58.195 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T02:28:58.195 INFO:tasks.cephadm.mds.c:Stopped mds.c 2026-02-13T02:28:58.195 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2026-02-13T02:28:58.196 DEBUG:teuthology.orchestra.run.trial107:> sudo systemctl stop ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.f 2026-02-13T02:28:58.501 INFO:journalctl@ceph.mds.f.trial107.stdout:Feb 13 02:28:58 trial107 systemd[1]: Stopping Ceph mds.f for 707dbb29-0882-11f1-9262-d404e6e7d460... 2026-02-13T02:28:58.501 INFO:journalctl@ceph.mds.f.trial107.stdout:Feb 13 02:28:58 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-f[55648]: 2026-02-13T02:28:58.326+0000 7f833d6a66c0 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2026-02-13T02:28:58.502 INFO:journalctl@ceph.mds.f.trial107.stdout:Feb 13 02:28:58 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-f[55648]: 2026-02-13T02:28:58.326+0000 7f833d6a66c0 -1 mds.f *** got signal Terminated *** 2026-02-13T02:28:59.377 DEBUG:teuthology.orchestra.run.trial107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.f.service' 2026-02-13T02:28:59.410 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T02:28:59.411 INFO:tasks.cephadm.mds.f:Stopped mds.f 2026-02-13T02:28:59.411 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2026-02-13T02:28:59.411 DEBUG:teuthology.orchestra.run.trial107:> sudo systemctl stop ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.i 2026-02-13T02:28:59.543 INFO:journalctl@ceph.mds.i.trial107.stdout:Feb 13 02:28:59 trial107 systemd[1]: Stopping Ceph mds.i for 707dbb29-0882-11f1-9262-d404e6e7d460... 2026-02-13T02:29:00.001 INFO:journalctl@ceph.mds.i.trial107.stdout:Feb 13 02:28:59 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-i[56293]: 2026-02-13T02:28:59.545+0000 7fe101a186c0 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2026-02-13T02:29:00.001 INFO:journalctl@ceph.mds.i.trial107.stdout:Feb 13 02:28:59 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-i[56293]: 2026-02-13T02:28:59.545+0000 7fe101a186c0 -1 mds.i *** got signal Terminated *** 2026-02-13T02:29:00.589 DEBUG:teuthology.orchestra.run.trial107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.i.service' 2026-02-13T02:29:00.621 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T02:29:00.622 INFO:tasks.cephadm.mds.i:Stopped mds.i 2026-02-13T02:29:00.622 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2026-02-13T02:29:00.622 DEBUG:teuthology.orchestra.run.trial107:> sudo systemctl stop ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.l 2026-02-13T02:29:00.757 INFO:journalctl@ceph.mds.l.trial107.stdout:Feb 13 02:29:00 trial107 systemd[1]: Stopping Ceph mds.l for 707dbb29-0882-11f1-9262-d404e6e7d460... 2026-02-13T02:29:01.251 INFO:journalctl@ceph.mds.l.trial107.stdout:Feb 13 02:29:00 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-l[55325]: 2026-02-13T02:29:00.759+0000 7f22802da6c0 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2026-02-13T02:29:01.251 INFO:journalctl@ceph.mds.l.trial107.stdout:Feb 13 02:29:00 trial107 ceph-707dbb29-0882-11f1-9262-d404e6e7d460-mds-l[55325]: 2026-02-13T02:29:00.759+0000 7f22802da6c0 -1 mds.l *** got signal Terminated *** 2026-02-13T02:29:01.781 DEBUG:teuthology.orchestra.run.trial107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-707dbb29-0882-11f1-9262-d404e6e7d460@mds.l.service' 2026-02-13T02:29:01.815 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-13T02:29:01.816 INFO:tasks.cephadm.mds.l:Stopped mds.l 2026-02-13T02:29:01.816 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 --force --keep-logs 2026-02-13T02:29:01.935 INFO:teuthology.orchestra.run.trial027.stdout:Deleting cluster with fsid: 707dbb29-0882-11f1-9262-d404e6e7d460 2026-02-13T02:29:03.764 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 --force --keep-logs 2026-02-13T02:29:03.885 INFO:teuthology.orchestra.run.trial028.stdout:Deleting cluster with fsid: 707dbb29-0882-11f1-9262-d404e6e7d460 2026-02-13T02:29:05.701 DEBUG:teuthology.orchestra.run.trial107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 --force --keep-logs 2026-02-13T02:29:05.822 INFO:teuthology.orchestra.run.trial107.stdout:Deleting cluster with fsid: 707dbb29-0882-11f1-9262-d404e6e7d460 2026-02-13T02:29:07.624 DEBUG:teuthology.orchestra.run.trial027:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2026-02-13T02:29:07.653 DEBUG:teuthology.orchestra.run.trial028:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2026-02-13T02:29:07.679 DEBUG:teuthology.orchestra.run.trial107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2026-02-13T02:29:07.709 INFO:tasks.cephadm:Archiving crash dumps... 2026-02-13T02:29:07.713 DEBUG:teuthology.misc:Transferring archived files from trial027:/var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/crash to /home/teuthworker/mnt/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47973/remote/trial027/crash 2026-02-13T02:29:07.714 DEBUG:teuthology.orchestra.run.trial027:> sudo tar c -f - -C /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/crash -- . 2026-02-13T02:29:07.739 INFO:teuthology.orchestra.run.trial027.stderr:tar: /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/crash: Cannot open: No such file or directory 2026-02-13T02:29:07.739 INFO:teuthology.orchestra.run.trial027.stderr:tar: Error is not recoverable: exiting now 2026-02-13T02:29:07.741 DEBUG:teuthology.misc:Transferring archived files from trial028:/var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/crash to /home/teuthworker/mnt/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47973/remote/trial028/crash 2026-02-13T02:29:07.743 DEBUG:teuthology.orchestra.run.trial028:> sudo tar c -f - -C /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/crash -- . 2026-02-13T02:29:07.767 INFO:teuthology.orchestra.run.trial028.stderr:tar: /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/crash: Cannot open: No such file or directory 2026-02-13T02:29:07.767 INFO:teuthology.orchestra.run.trial028.stderr:tar: Error is not recoverable: exiting now 2026-02-13T02:29:07.770 DEBUG:teuthology.misc:Transferring archived files from trial107:/var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/crash to /home/teuthworker/mnt/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47973/remote/trial107/crash 2026-02-13T02:29:07.771 DEBUG:teuthology.orchestra.run.trial107:> sudo tar c -f - -C /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/crash -- . 2026-02-13T02:29:07.796 INFO:teuthology.orchestra.run.trial107.stderr:tar: /var/lib/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/crash: Cannot open: No such file or directory 2026-02-13T02:29:07.796 INFO:teuthology.orchestra.run.trial107.stderr:tar: Error is not recoverable: exiting now 2026-02-13T02:29:07.797 INFO:tasks.cephadm:Checking cluster log for badness... 2026-02-13T02:29:07.797 DEBUG:teuthology.orchestra.run.trial027:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph.log | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | grep -E -v FS_DEGRADED | grep -E -v 'fs.*is degraded' | grep -E -v 'filesystem is degraded' | grep -E -v FS_INLINE_DATA_DEPRECATED | grep -E -v FS_WITH_FAILED_MDS | grep -E -v MDS_ALL_DOWN | grep -E -v 'filesystem is offline' | grep -E -v 'is offline because no MDS' | grep -E -v MDS_DAMAGE | grep -E -v MDS_DEGRADED | grep -E -v MDS_FAILED | grep -E -v MDS_INSUFFICIENT_STANDBY | grep -E -v 'insufficient standby MDS daemons available' | grep -E -v MDS_UP_LESS_THAN_MAX | grep -E -v 'online, but wants' | grep -E -v 'filesystem is online with fewer MDS than max_mds' | grep -E -v POOL_APP_NOT_ENABLED | grep -E -v 'do not have an application enabled' | grep -E -v 'overall HEALTH_' | grep -E -v 'Replacing daemon' | grep -E -v 'deprecated feature inline_data' | grep -E -v BLUESTORE_SLOW_OP_ALERT | grep -E -v 'slow operation indications in BlueStore' | grep -E -v 'experiencing slow operations in BlueStore' | grep -E -v MGR_MODULE_ERROR | grep -E -v OSD_DOWN | grep -E -v 'osd.* is down' | grep -E -v 'overall HEALTH_' | grep -E -v '\(OSD_DOWN\)' | grep -E -v '\(OSD_' | grep -E -v 'but it is still running' | grep -E -v 'is not responding' | grep -E -v 'osds down' | grep -E -v 'osd.* is down' | grep -E -v PG_AVAILABILITY | grep -E -v PG_DEGRADED | grep -E -v 'Reduced data availability' | grep -E -v 'Degraded data redundancy' | grep -E -v 'pg .* is stuck inactive' | grep -E -v 'pg .* is .*degraded' | grep -E -v 'slow metadata IO' | grep -E -v SLOW_OPS | grep -E -v 'slow request' | head -n 1 2026-02-13T02:29:07.821 INFO:teuthology.orchestra.run.trial027.stdout:2026-02-13T02:20:31.306060+0000 mon.a (mon.0) 301 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:29:07.823 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2026-02-13T02:29:07.823 DEBUG:teuthology.orchestra.run.trial027:> sudo grep -E '\[SEC\]' /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph.log | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | grep -E -v FS_DEGRADED | grep -E -v 'fs.*is degraded' | grep -E -v 'filesystem is degraded' | grep -E -v FS_INLINE_DATA_DEPRECATED | grep -E -v FS_WITH_FAILED_MDS | grep -E -v MDS_ALL_DOWN | grep -E -v 'filesystem is offline' | grep -E -v 'is offline because no MDS' | grep -E -v MDS_DAMAGE | grep -E -v MDS_DEGRADED | grep -E -v MDS_FAILED | grep -E -v MDS_INSUFFICIENT_STANDBY | grep -E -v 'insufficient standby MDS daemons available' | grep -E -v MDS_UP_LESS_THAN_MAX | grep -E -v 'online, but wants' | grep -E -v 'filesystem is online with fewer MDS than max_mds' | grep -E -v POOL_APP_NOT_ENABLED | grep -E -v 'do not have an application enabled' | grep -E -v 'overall HEALTH_' | grep -E -v 'Replacing daemon' | grep -E -v 'deprecated feature inline_data' | grep -E -v BLUESTORE_SLOW_OP_ALERT | grep -E -v 'slow operation indications in BlueStore' | grep -E -v 'experiencing slow operations in BlueStore' | grep -E -v MGR_MODULE_ERROR | grep -E -v OSD_DOWN | grep -E -v 'osd.* is down' | grep -E -v 'overall HEALTH_' | grep -E -v '\(OSD_DOWN\)' | grep -E -v '\(OSD_' | grep -E -v 'but it is still running' | grep -E -v 'is not responding' | grep -E -v 'osds down' | grep -E -v 'osd.* is down' | grep -E -v PG_AVAILABILITY | grep -E -v PG_DEGRADED | grep -E -v 'Reduced data availability' | grep -E -v 'Degraded data redundancy' | grep -E -v 'pg .* is stuck inactive' | grep -E -v 'pg .* is .*degraded' | grep -E -v 'slow metadata IO' | grep -E -v SLOW_OPS | grep -E -v 'slow request' | head -n 1 2026-02-13T02:29:07.888 DEBUG:teuthology.orchestra.run.trial027:> sudo grep -E '\[ERR\]' /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph.log | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | grep -E -v FS_DEGRADED | grep -E -v 'fs.*is degraded' | grep -E -v 'filesystem is degraded' | grep -E -v FS_INLINE_DATA_DEPRECATED | grep -E -v FS_WITH_FAILED_MDS | grep -E -v MDS_ALL_DOWN | grep -E -v 'filesystem is offline' | grep -E -v 'is offline because no MDS' | grep -E -v MDS_DAMAGE | grep -E -v MDS_DEGRADED | grep -E -v MDS_FAILED | grep -E -v MDS_INSUFFICIENT_STANDBY | grep -E -v 'insufficient standby MDS daemons available' | grep -E -v MDS_UP_LESS_THAN_MAX | grep -E -v 'online, but wants' | grep -E -v 'filesystem is online with fewer MDS than max_mds' | grep -E -v POOL_APP_NOT_ENABLED | grep -E -v 'do not have an application enabled' | grep -E -v 'overall HEALTH_' | grep -E -v 'Replacing daemon' | grep -E -v 'deprecated feature inline_data' | grep -E -v BLUESTORE_SLOW_OP_ALERT | grep -E -v 'slow operation indications in BlueStore' | grep -E -v 'experiencing slow operations in BlueStore' | grep -E -v MGR_MODULE_ERROR | grep -E -v OSD_DOWN | grep -E -v 'osd.* is down' | grep -E -v 'overall HEALTH_' | grep -E -v '\(OSD_DOWN\)' | grep -E -v '\(OSD_' | grep -E -v 'but it is still running' | grep -E -v 'is not responding' | grep -E -v 'osds down' | grep -E -v 'osd.* is down' | grep -E -v PG_AVAILABILITY | grep -E -v PG_DEGRADED | grep -E -v 'Reduced data availability' | grep -E -v 'Degraded data redundancy' | grep -E -v 'pg .* is stuck inactive' | grep -E -v 'pg .* is .*degraded' | grep -E -v 'slow metadata IO' | grep -E -v SLOW_OPS | grep -E -v 'slow request' | head -n 1 2026-02-13T02:29:07.953 DEBUG:teuthology.orchestra.run.trial027:> sudo grep -E '\[WRN\]' /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph.log | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | grep -E -v FS_DEGRADED | grep -E -v 'fs.*is degraded' | grep -E -v 'filesystem is degraded' | grep -E -v FS_INLINE_DATA_DEPRECATED | grep -E -v FS_WITH_FAILED_MDS | grep -E -v MDS_ALL_DOWN | grep -E -v 'filesystem is offline' | grep -E -v 'is offline because no MDS' | grep -E -v MDS_DAMAGE | grep -E -v MDS_DEGRADED | grep -E -v MDS_FAILED | grep -E -v MDS_INSUFFICIENT_STANDBY | grep -E -v 'insufficient standby MDS daemons available' | grep -E -v MDS_UP_LESS_THAN_MAX | grep -E -v 'online, but wants' | grep -E -v 'filesystem is online with fewer MDS than max_mds' | grep -E -v POOL_APP_NOT_ENABLED | grep -E -v 'do not have an application enabled' | grep -E -v 'overall HEALTH_' | grep -E -v 'Replacing daemon' | grep -E -v 'deprecated feature inline_data' | grep -E -v BLUESTORE_SLOW_OP_ALERT | grep -E -v 'slow operation indications in BlueStore' | grep -E -v 'experiencing slow operations in BlueStore' | grep -E -v MGR_MODULE_ERROR | grep -E -v OSD_DOWN | grep -E -v 'osd.* is down' | grep -E -v 'overall HEALTH_' | grep -E -v '\(OSD_DOWN\)' | grep -E -v '\(OSD_' | grep -E -v 'but it is still running' | grep -E -v 'is not responding' | grep -E -v 'osds down' | grep -E -v 'osd.* is down' | grep -E -v PG_AVAILABILITY | grep -E -v PG_DEGRADED | grep -E -v 'Reduced data availability' | grep -E -v 'Degraded data redundancy' | grep -E -v 'pg .* is stuck inactive' | grep -E -v 'pg .* is .*degraded' | grep -E -v 'slow metadata IO' | grep -E -v SLOW_OPS | grep -E -v 'slow request' | head -n 1 2026-02-13T02:29:08.016 INFO:teuthology.orchestra.run.trial027.stdout:2026-02-13T02:20:31.306060+0000 mon.a (mon.0) 301 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL) 2026-02-13T02:29:08.017 INFO:tasks.cephadm:Compressing logs... 2026-02-13T02:29:08.017 DEBUG:teuthology.orchestra.run.trial027:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-13T02:29:08.060 DEBUG:teuthology.orchestra.run.trial028:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-13T02:29:08.063 DEBUG:teuthology.orchestra.run.trial107:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-13T02:29:08.083 INFO:teuthology.orchestra.run.trial027.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2026-02-13T02:29:08.083 INFO:teuthology.orchestra.run.trial027.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2026-02-13T02:29:08.083 INFO:teuthology.orchestra.run.trial027.stderr:gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mon.a.log 2026-02-13T02:29:08.083 INFO:teuthology.orchestra.run.trial027.stderr:gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.d.log 2026-02-13T02:29:08.088 INFO:teuthology.orchestra.run.trial027.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.j.log 2026-02-13T02:29:08.088 INFO:teuthology.orchestra.run.trial027.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.0.log 2026-02-13T02:29:08.088 INFO:teuthology.orchestra.run.trial027.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.3.log 2026-02-13T02:29:08.088 INFO:teuthology.orchestra.run.trial027.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph.cephadm.log 2026-02-13T02:29:08.088 INFO:teuthology.orchestra.run.trial027.stderr:gzip -5 --verbose --/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.3.log: /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.a.log 2026-02-13T02:29:08.088 INFO:teuthology.orchestra.run.trial027.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.g.log 2026-02-13T02:29:08.088 INFO:teuthology.orchestra.run.trial027.stderr: 43.6% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph.cephadm.log.gz 2026-02-13T02:29:08.088 INFO:teuthology.orchestra.run.trial027.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph.audit.log 2026-02-13T02:29:08.088 INFO:teuthology.orchestra.run.trial027.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-volume.log 2026-02-13T02:29:08.088 INFO:teuthology.orchestra.run.trial027.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.6.log 2026-02-13T02:29:08.089 INFO:teuthology.orchestra.run.trial027.stderr: 95.0% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.d.log.gz 2026-02-13T02:29:08.089 INFO:teuthology.orchestra.run.trial027.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mgr.x.log 2026-02-13T02:29:08.089 INFO:teuthology.orchestra.run.trial027.stderr: 95.0% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.j.log.gz 2026-02-13T02:29:08.089 INFO:teuthology.orchestra.run.trial027.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.6.log: 94.7% -- replaced with /var/log/ceph/cephadm.log.gz 2026-02-13T02:29:08.089 INFO:teuthology.orchestra.run.trial027.stderr:gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph.log 2026-02-13T02:29:08.089 INFO:teuthology.orchestra.run.trial027.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.9.log 2026-02-13T02:29:08.089 INFO:teuthology.orchestra.run.trial027.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph.log: 94.8% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.g.log.gz 2026-02-13T02:29:08.089 INFO:teuthology.orchestra.run.trial027.stderr: 94.8% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.a.log.gz 2026-02-13T02:29:08.089 INFO:teuthology.orchestra.run.trial027.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.9.log: 90.9% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph.audit.log.gz 2026-02-13T02:29:08.089 INFO:teuthology.orchestra.run.trial027.stderr: 87.8% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph.log.gz 2026-02-13T02:29:08.095 INFO:teuthology.orchestra.run.trial027.stderr: 93.3% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-volume.log.gz 2026-02-13T02:29:08.099 INFO:teuthology.orchestra.run.trial107.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2026-02-13T02:29:08.099 INFO:teuthology.orchestra.run.trial107.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2026-02-13T02:29:08.099 INFO:teuthology.orchestra.run.trial107.stderr:gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.2.log 2026-02-13T02:29:08.099 INFO:teuthology.orchestra.run.trial107.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.i.log 2026-02-13T02:29:08.099 INFO:teuthology.orchestra.run.trial107.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mon.c.log 2026-02-13T02:29:08.099 INFO:teuthology.orchestra.run.trial107.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.f.log 2026-02-13T02:29:08.099 INFO:teuthology.orchestra.run.trial107.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.5.log 2026-02-13T02:29:08.099 INFO:teuthology.orchestra.run.trial107.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.11.log 2026-02-13T02:29:08.099 INFO:teuthology.orchestra.run.trial107.stderr: 93.6% -- replaced with /var/log/ceph/cephadm.log.gz 2026-02-13T02:29:08.099 INFO:teuthology.orchestra.run.trial107.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph.audit.log 2026-02-13T02:29:08.100 INFO:teuthology.orchestra.run.trial107.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-volume.log 2026-02-13T02:29:08.100 INFO:teuthology.orchestra.run.trial107.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mgr.z.log 2026-02-13T02:29:08.100 INFO:teuthology.orchestra.run.trial107.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.l.log 2026-02-13T02:29:08.100 INFO:teuthology.orchestra.run.trial107.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.c.log 2026-02-13T02:29:08.100 INFO:teuthology.orchestra.run.trial107.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.l.log: 94.8% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.i.log.gz 2026-02-13T02:29:08.100 INFO:teuthology.orchestra.run.trial107.stderr:gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph.log 2026-02-13T02:29:08.100 INFO:teuthology.orchestra.run.trial107.stderr: 94.8% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.f.log.gz 2026-02-13T02:29:08.100 INFO:teuthology.orchestra.run.trial107.stderr:gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.8.log 2026-02-13T02:29:08.100 INFO:teuthology.orchestra.run.trial107.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.c.log: 91.1% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph.audit.log.gz 2026-02-13T02:29:08.100 INFO:teuthology.orchestra.run.trial107.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph.log: /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.8.log: 87.9% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph.log.gz 2026-02-13T02:29:08.100 INFO:teuthology.orchestra.run.trial107.stderr: 95.0% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.l.log.gz 2026-02-13T02:29:08.101 INFO:teuthology.orchestra.run.trial107.stderr: 94.9% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.c.log.gz 2026-02-13T02:29:08.101 INFO:teuthology.orchestra.run.trial107.stderr: 92.2% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mgr.z.log.gz 2026-02-13T02:29:08.101 INFO:teuthology.orchestra.run.trial107.stderr: 93.7% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-volume.log.gz 2026-02-13T02:29:08.101 INFO:teuthology.orchestra.run.trial028.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2026-02-13T02:29:08.101 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2026-02-13T02:29:08.102 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mgr.y.log 2026-02-13T02:29:08.102 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.b.log 2026-02-13T02:29:08.102 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.h.log 2026-02-13T02:29:08.102 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mgr.y.log: /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.7.log 2026-02-13T02:29:08.102 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.h.log: gzip 92.8% -5 -- replaced with /var/log/ceph/cephadm.log.gz --verbose 2026-02-13T02:29:08.102 INFO:teuthology.orchestra.run.trial028.stderr: -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.10.log 2026-02-13T02:29:08.102 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.4.log 2026-02-13T02:29:08.102 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph.audit.log 2026-02-13T02:29:08.102 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mon.b.log 2026-02-13T02:29:08.102 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-volume.log 2026-02-13T02:29:08.102 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph.audit.log: /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.1.log 2026-02-13T02:29:08.103 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph.log 2026-02-13T02:29:08.103 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.k.log 2026-02-13T02:29:08.103 INFO:teuthology.orchestra.run.trial028.stderr: 92.3% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mgr.y.log.gz 2026-02-13T02:29:08.103 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph.log: gzip -5 --verbose -- /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.e.log 2026-02-13T02:29:08.103 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.k.log: 91.0% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph.audit.log.gz 2026-02-13T02:29:08.103 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.e.log: 87.9% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph.log.gz 2026-02-13T02:29:08.103 INFO:teuthology.orchestra.run.trial028.stderr: 94.8% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.e.log.gz 2026-02-13T02:29:08.103 INFO:teuthology.orchestra.run.trial028.stderr: 94.9% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.k.log.gz 2026-02-13T02:29:08.103 INFO:teuthology.orchestra.run.trial028.stderr: 93.3% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-volume.log.gz 2026-02-13T02:29:08.103 INFO:teuthology.orchestra.run.trial028.stderr: 94.5% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.h.log.gz 2026-02-13T02:29:08.112 INFO:teuthology.orchestra.run.trial028.stderr: 93.5% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mds.b.log.gz 2026-02-13T02:29:08.171 INFO:teuthology.orchestra.run.trial028.stderr: 92.3% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mon.b.log.gz 2026-02-13T02:29:08.174 INFO:teuthology.orchestra.run.trial107.stderr: 92.3% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mon.c.log.gz 2026-02-13T02:29:08.175 INFO:teuthology.orchestra.run.trial027.stderr: 90.2% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mgr.x.log.gz 2026-02-13T02:29:08.217 INFO:teuthology.orchestra.run.trial107.stderr: 92.9% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.11.log.gz 2026-02-13T02:29:08.220 INFO:teuthology.orchestra.run.trial027.stderr: 92.3% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.9.log.gz 2026-02-13T02:29:08.222 INFO:teuthology.orchestra.run.trial028.stderr: 92.4% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.4.log.gz 2026-02-13T02:29:08.232 INFO:teuthology.orchestra.run.trial027.stderr: 92.4% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.6.log.gz 2026-02-13T02:29:08.243 INFO:teuthology.orchestra.run.trial027.stderr: 92.3% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.3.log.gz 2026-02-13T02:29:08.244 INFO:teuthology.orchestra.run.trial027.stderr: 92.5% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.0.log.gz 2026-02-13T02:29:08.244 INFO:teuthology.orchestra.run.trial107.stderr: 92.5% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.8.log.gz 2026-02-13T02:29:08.247 INFO:teuthology.orchestra.run.trial028.stderr: 92.5% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.7.log.gz 2026-02-13T02:29:08.268 INFO:teuthology.orchestra.run.trial107.stderr: 92.1% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.5.log.gz 2026-02-13T02:29:08.271 INFO:teuthology.orchestra.run.trial107.stderr: 92.2% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.2.log.gz 2026-02-13T02:29:08.274 INFO:teuthology.orchestra.run.trial107.stderr: 2026-02-13T02:29:08.274 INFO:teuthology.orchestra.run.trial107.stderr:real 0m0.193s 2026-02-13T02:29:08.274 INFO:teuthology.orchestra.run.trial107.stderr:user 0m0.719s 2026-02-13T02:29:08.274 INFO:teuthology.orchestra.run.trial107.stderr:sys 0m0.049s 2026-02-13T02:29:08.283 INFO:teuthology.orchestra.run.trial028.stderr: 92.1% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.10.log.gz 2026-02-13T02:29:08.290 INFO:teuthology.orchestra.run.trial028.stderr: 92.2% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-osd.1.log.gz 2026-02-13T02:29:08.291 INFO:teuthology.orchestra.run.trial028.stderr: 2026-02-13T02:29:08.291 INFO:teuthology.orchestra.run.trial028.stderr:real 0m0.215s 2026-02-13T02:29:08.291 INFO:teuthology.orchestra.run.trial028.stderr:user 0m0.801s 2026-02-13T02:29:08.291 INFO:teuthology.orchestra.run.trial028.stderr:sys 0m0.056s 2026-02-13T02:29:08.376 INFO:teuthology.orchestra.run.trial027.stderr: 90.2% -- replaced with /var/log/ceph/707dbb29-0882-11f1-9262-d404e6e7d460/ceph-mon.a.log.gz 2026-02-13T02:29:08.377 INFO:teuthology.orchestra.run.trial027.stderr: 2026-02-13T02:29:08.378 INFO:teuthology.orchestra.run.trial027.stderr:real 0m0.304s 2026-02-13T02:29:08.378 INFO:teuthology.orchestra.run.trial027.stderr:user 0m0.948s 2026-02-13T02:29:08.378 INFO:teuthology.orchestra.run.trial027.stderr:sys 0m0.075s 2026-02-13T02:29:08.379 INFO:tasks.cephadm:Archiving logs... 2026-02-13T02:29:08.379 DEBUG:teuthology.misc:Transferring archived files from trial027:/var/log/ceph to /home/teuthworker/mnt/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47973/remote/trial027/log 2026-02-13T02:29:08.381 DEBUG:teuthology.orchestra.run.trial027:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-13T02:29:08.623 DEBUG:teuthology.misc:Transferring archived files from trial028:/var/log/ceph to /home/teuthworker/mnt/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47973/remote/trial028/log 2026-02-13T02:29:08.625 DEBUG:teuthology.orchestra.run.trial028:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-13T02:29:08.752 DEBUG:teuthology.misc:Transferring archived files from trial107:/var/log/ceph to /home/teuthworker/mnt/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47973/remote/trial107/log 2026-02-13T02:29:08.752 DEBUG:teuthology.orchestra.run.trial107:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-13T02:29:08.856 INFO:tasks.cephadm:Removing cluster... 2026-02-13T02:29:08.856 DEBUG:teuthology.orchestra.run.trial027:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 --force 2026-02-13T02:29:08.974 INFO:teuthology.orchestra.run.trial027.stdout:Deleting cluster with fsid: 707dbb29-0882-11f1-9262-d404e6e7d460 2026-02-13T02:29:09.138 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 --force 2026-02-13T02:29:09.257 INFO:teuthology.orchestra.run.trial028.stdout:Deleting cluster with fsid: 707dbb29-0882-11f1-9262-d404e6e7d460 2026-02-13T02:29:09.417 DEBUG:teuthology.orchestra.run.trial107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 707dbb29-0882-11f1-9262-d404e6e7d460 --force 2026-02-13T02:29:09.536 INFO:teuthology.orchestra.run.trial107.stdout:Deleting cluster with fsid: 707dbb29-0882-11f1-9262-d404e6e7d460 2026-02-13T02:29:09.701 INFO:tasks.cephadm:Removing cephadm ... 2026-02-13T02:29:09.701 DEBUG:teuthology.orchestra.run.trial027:> rm -rf /home/ubuntu/cephtest/cephadm 2026-02-13T02:29:09.720 DEBUG:teuthology.orchestra.run.trial028:> rm -rf /home/ubuntu/cephtest/cephadm 2026-02-13T02:29:09.738 DEBUG:teuthology.orchestra.run.trial107:> rm -rf /home/ubuntu/cephtest/cephadm 2026-02-13T02:29:09.755 INFO:tasks.cephadm:Teardown complete 2026-02-13T02:29:09.755 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_a2c505dae37458c5537fa44f1320bd04eef25bb1/teuthology/run_tasks.py", line 112, in run_tasks manager.__enter__() File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_batrick_ceph_cfcf307502511ba51c39a8b9085c2fcde505d421/qa/tasks/cephadm.py", line 2025, in task healthy(ctx=ctx, config=config) File "/home/teuthworker/src/github.com_batrick_ceph_cfcf307502511ba51c39a8b9085c2fcde505d421/qa/tasks/ceph.py", line 1557, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/github.com_batrick_ceph_cfcf307502511ba51c39a8b9085c2fcde505d421/qa/tasks/ceph_manager.py", line 3393, in wait_until_healthy assert time.time() - start < timeout, \ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AssertionError: timeout expired in wait_until_healthy 2026-02-13T02:29:09.757 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2026-02-13T02:29:09.774 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-13T02:29:09.789 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-13T02:29:09.789 DEBUG:teuthology.orchestra.run.trial027:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-13T02:29:09.792 DEBUG:teuthology.orchestra.run.trial028:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-13T02:29:09.794 DEBUG:teuthology.orchestra.run.trial107:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-13T02:29:10.032 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-13T02:29:10.043 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-13T02:29:10.043 DEBUG:teuthology.orchestra.run.trial027:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-13T02:29:10.045 DEBUG:teuthology.orchestra.run.trial028:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-13T02:29:10.047 DEBUG:teuthology.orchestra.run.trial107:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-13T02:29:10.060 INFO:teuthology.orchestra.run.trial027.stderr:bash: line 1: ntpq: command not found 2026-02-13T02:29:10.061 INFO:teuthology.orchestra.run.trial028.stderr:bash: line 1: ntpq: command not found 2026-02-13T02:29:10.062 INFO:teuthology.orchestra.run.trial107.stderr:bash: line 1: ntpq: command not found 2026-02-13T02:29:10.155 INFO:teuthology.orchestra.run.trial027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-13T02:29:10.155 INFO:teuthology.orchestra.run.trial027.stdout:=============================================================================== 2026-02-13T02:29:10.155 INFO:teuthology.orchestra.run.trial027.stdout:^- 23.186.168.131 2 6 377 63 -6747us[-6772us] +/- 64ms 2026-02-13T02:29:10.155 INFO:teuthology.orchestra.run.trial027.stdout:^* nyc2.us.ntp.li 2 6 377 2 -65us[ -91us] +/- 2786us 2026-02-13T02:29:10.155 INFO:teuthology.orchestra.run.trial027.stdout:^- 172-234-44-141.ip.linode> 2 6 377 2 +14ms[ +14ms] +/- 85ms 2026-02-13T02:29:10.155 INFO:teuthology.orchestra.run.trial027.stdout:^- chi2.us.ntp.li 2 6 377 63 -23us[ -48us] +/- 11ms 2026-02-13T02:29:10.156 INFO:teuthology.orchestra.run.trial028.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-13T02:29:10.156 INFO:teuthology.orchestra.run.trial028.stdout:=============================================================================== 2026-02-13T02:29:10.156 INFO:teuthology.orchestra.run.trial028.stdout:^+ 172-232-15-202.ip.linode> 2 6 377 62 -3211us[-3071us] +/- 77ms 2026-02-13T02:29:10.156 INFO:teuthology.orchestra.run.trial028.stdout:^* edge-anycast.ntp00.hosti> 2 6 377 0 +4634us[+4774us] +/- 43ms 2026-02-13T02:29:10.156 INFO:teuthology.orchestra.run.trial028.stdout:^+ 69-10-208-170.rainiercon> 2 6 377 64 -3566us[-3426us] +/- 77ms 2026-02-13T02:29:10.156 INFO:teuthology.orchestra.run.trial028.stdout:^+ 172-104-28-175.ip.linode> 2 6 377 3 -1749us[-1609us] +/- 57ms 2026-02-13T02:29:10.163 INFO:teuthology.orchestra.run.trial107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-13T02:29:10.163 INFO:teuthology.orchestra.run.trial107.stdout:=============================================================================== 2026-02-13T02:29:10.163 INFO:teuthology.orchestra.run.trial107.stdout:^+ 66.42.71.197.vultruserco> 3 6 377 63 +2428us[+2428us] +/- 44ms 2026-02-13T02:29:10.163 INFO:teuthology.orchestra.run.trial107.stdout:^+ 23.186.168.133 2 6 267 64 -2810us[-2804us] +/- 64ms 2026-02-13T02:29:10.163 INFO:teuthology.orchestra.run.trial107.stdout:^* time.walb.tech 2 6 377 64 +2468us[+2474us] +/- 40ms 2026-02-13T02:29:10.163 INFO:teuthology.orchestra.run.trial107.stdout:^+ ntp.maxhost.io 2 6 277 64 -4539us[-4532us] +/- 67ms 2026-02-13T02:29:10.163 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-13T02:29:10.175 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-13T02:29:10.175 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-13T02:29:10.190 DEBUG:teuthology.orchestra.run.trial027:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-13T02:29:10.247 DEBUG:teuthology.orchestra.run.trial028:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-13T02:29:10.292 DEBUG:teuthology.orchestra.run.trial107:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-13T02:29:10.338 DEBUG:teuthology.orchestra.run.trial027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-13T02:29:10.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T02:29:10.366 DEBUG:teuthology.orchestra.run.trial028:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-13T02:29:10.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T02:29:10.392 DEBUG:teuthology.orchestra.run.trial107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-13T02:29:10.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T02:29:10.420 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-13T02:29:10.435 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-13T02:29:10.450 INFO:teuthology.task.internal:Duration was 727.283111 seconds 2026-02-13T02:29:10.451 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-13T02:29:10.466 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-13T02:29:10.467 DEBUG:teuthology.orchestra.run.trial027:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-13T02:29:10.469 DEBUG:teuthology.orchestra.run.trial028:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-13T02:29:10.472 DEBUG:teuthology.orchestra.run.trial107:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-13T02:29:10.504 INFO:teuthology.orchestra.run.trial027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-13T02:29:10.506 INFO:teuthology.orchestra.run.trial028.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-13T02:29:10.508 INFO:teuthology.orchestra.run.trial107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-13T02:29:10.944 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-13T02:29:10.944 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial027.front.sepia.ceph.com 2026-02-13T02:29:10.944 DEBUG:teuthology.orchestra.run.trial027:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-13T02:29:11.003 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial028.front.sepia.ceph.com 2026-02-13T02:29:11.003 DEBUG:teuthology.orchestra.run.trial028:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-13T02:29:11.025 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial107.front.sepia.ceph.com 2026-02-13T02:29:11.025 DEBUG:teuthology.orchestra.run.trial107:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-13T02:29:11.045 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-13T02:29:11.045 DEBUG:teuthology.orchestra.run.trial027:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-13T02:29:11.047 DEBUG:teuthology.orchestra.run.trial028:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-13T02:29:11.067 DEBUG:teuthology.orchestra.run.trial107:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-13T02:29:11.294 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-13T02:29:11.294 DEBUG:teuthology.orchestra.run.trial027:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-13T02:29:11.295 DEBUG:teuthology.orchestra.run.trial028:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-13T02:29:11.297 DEBUG:teuthology.orchestra.run.trial107:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-13T02:29:11.317 INFO:teuthology.orchestra.run.trial027.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-13T02:29:11.318 INFO:teuthology.orchestra.run.trial027.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-13T02:29:11.318 INFO:teuthology.orchestra.run.trial027.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-13T02:29:11.318 INFO:teuthology.orchestra.run.trial027.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-13T02:29:11.318 INFO:teuthology.orchestra.run.trial027.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-13T02:29:11.320 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-13T02:29:11.320 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-13T02:29:11.321 INFO:teuthology.orchestra.run.trial028.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-13T02:29:11.321 INFO:teuthology.orchestra.run.trial028.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-13T02:29:11.321 INFO:teuthology.orchestra.run.trial028.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-13T02:29:11.322 INFO:teuthology.orchestra.run.trial107.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-13T02:29:11.322 INFO:teuthology.orchestra.run.trial107.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-13T02:29:11.322 INFO:teuthology.orchestra.run.trial107.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-13T02:29:11.322 INFO:teuthology.orchestra.run.trial107.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-13T02:29:11.322 INFO:teuthology.orchestra.run.trial107.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-13T02:29:11.333 INFO:teuthology.orchestra.run.trial028.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 90.4% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-13T02:29:11.336 INFO:teuthology.orchestra.run.trial107.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 90.5% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-13T02:29:11.365 INFO:teuthology.orchestra.run.trial027.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 92.2% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-13T02:29:11.367 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-13T02:29:11.395 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-13T02:29:11.395 DEBUG:teuthology.orchestra.run.trial027:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-13T02:29:11.431 DEBUG:teuthology.orchestra.run.trial028:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-13T02:29:11.458 DEBUG:teuthology.orchestra.run.trial107:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-13T02:29:11.484 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-13T02:29:11.501 DEBUG:teuthology.orchestra.run.trial027:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-13T02:29:11.503 DEBUG:teuthology.orchestra.run.trial028:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-13T02:29:11.507 DEBUG:teuthology.orchestra.run.trial107:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-13T02:29:11.527 INFO:teuthology.orchestra.run.trial027.stdout:kernel.core_pattern = core 2026-02-13T02:29:11.530 INFO:teuthology.orchestra.run.trial028.stdout:kernel.core_pattern = core 2026-02-13T02:29:11.548 INFO:teuthology.orchestra.run.trial107.stdout:kernel.core_pattern = core 2026-02-13T02:29:11.562 DEBUG:teuthology.orchestra.run.trial027:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-13T02:29:11.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T02:29:11.595 DEBUG:teuthology.orchestra.run.trial028:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-13T02:29:11.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T02:29:11.611 DEBUG:teuthology.orchestra.run.trial107:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-13T02:29:11.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-13T02:29:11.626 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-13T02:29:11.643 INFO:teuthology.task.internal:Transferring archived files... 2026-02-13T02:29:11.644 DEBUG:teuthology.misc:Transferring archived files from trial027:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47973/remote/trial027 2026-02-13T02:29:11.644 DEBUG:teuthology.orchestra.run.trial027:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-13T02:29:11.688 DEBUG:teuthology.misc:Transferring archived files from trial028:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47973/remote/trial028 2026-02-13T02:29:11.688 DEBUG:teuthology.orchestra.run.trial028:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-13T02:29:11.724 DEBUG:teuthology.misc:Transferring archived files from trial107:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47973/remote/trial107 2026-02-13T02:29:11.724 DEBUG:teuthology.orchestra.run.trial107:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-13T02:29:11.762 INFO:teuthology.task.internal:Removing archive directory... 2026-02-13T02:29:11.762 DEBUG:teuthology.orchestra.run.trial027:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-13T02:29:11.765 DEBUG:teuthology.orchestra.run.trial028:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-13T02:29:11.768 DEBUG:teuthology.orchestra.run.trial107:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-13T02:29:11.808 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-13T02:29:11.827 INFO:teuthology.task.internal:Not uploading archives. 2026-02-13T02:29:11.828 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-13T02:29:11.845 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-13T02:29:11.845 DEBUG:teuthology.orchestra.run.trial027:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-13T02:29:11.848 DEBUG:teuthology.orchestra.run.trial028:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-13T02:29:11.851 DEBUG:teuthology.orchestra.run.trial107:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-13T02:29:11.863 INFO:teuthology.orchestra.run.trial027.stdout: 791883 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 13 02:29 /home/ubuntu/cephtest 2026-02-13T02:29:11.865 INFO:teuthology.orchestra.run.trial028.stdout: 786525 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 13 02:29 /home/ubuntu/cephtest 2026-02-13T02:29:11.867 INFO:teuthology.orchestra.run.trial107.stdout: 786525 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 13 02:29 /home/ubuntu/cephtest 2026-02-13T02:29:11.868 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-13T02:29:11.886 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-13T02:29:11.928 INFO:teuthology.run:Summary data: description: fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/suites/dbench}} duration: 727.2831106185913 failure_reason: '"2026-02-13T02:20:31.306060+0000 mon.a (mon.0) 301 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log' flavor: default owner: scheduled_pdonnell@soko04.front.sepia.ceph.com sentry_event: null status: fail success: false 2026-02-13T02:29:11.929 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-13T02:29:12.062 INFO:teuthology.run:FAIL